-- Logs begin at Fri 2022-06-03 08:58:30 UTC, end at Fri 2022-06-03 10:15:24 UTC. -- Jun 03 09:36:26.307118 dv-cinder-844563-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 03 09:36:26.316373 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 3 09:36:26 2022] *** Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: nodename: dv-cinder-844563-1-iscsi Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: machine: x86_64 Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: clock source: unix Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: pcre jit disabled Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: detected number of CPU cores: 4 Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: current working directory: / Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: detected binary path: /usr/bin/uwsgi-core Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: your processes number limit is 47680 Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: your memory page size is 4096 bytes Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: detected max file descriptor number: 2048 Jun 03 09:36:26.318450 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: lock engine: pthread robust mutexes Jun 03 09:36:26.319236 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: thunder lock: enabled Jun 03 09:36:26.319236 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 03 09:36:26.319236 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 03 09:36:26.363127 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: Python main interpreter initialized at 0x557066a389c0 Jun 03 09:36:26.363127 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: python threads support enabled Jun 03 09:36:26.363127 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: your server socket listen backlog is limited to 100 connections Jun 03 09:36:26.363127 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: your mercy for graceful operations on workers is 90 seconds Jun 03 09:36:26.363491 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: mapped 403077 bytes (393 KB) for 2 cores Jun 03 09:36:26.363580 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: *** Operational MODE: preforking *** Jun 03 09:36:26.363853 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: *** uWSGI is running in multiple interpreter mode *** Jun 03 09:36:26.363853 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: spawned uWSGI master process (pid: 87014) Jun 03 09:36:26.364050 dv-cinder-844563-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 03 09:36:26.366054 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: spawned uWSGI worker 1 (pid: 87025, cores: 1) Jun 03 09:36:26.366054 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: spawned uWSGI worker 2 (pid: 87026, cores: 1) Jun 03 09:36:26.366054 dv-cinder-844563-1-iscsi devstack@keystone.service[87014]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 03 09:36:32.096559 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 09:36:32.096559 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: from cryptography.utils import int_from_bytes Jun 03 09:36:32.097318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 09:36:32.097318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: from cryptography.utils import int_from_bytes Jun 03 09:36:32.097318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 09:36:32.097318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: from cryptography.utils import int_from_bytes Jun 03 09:36:32.097875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 09:36:32.097875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: from cryptography.utils import int_from_bytes Jun 03 09:36:32.146930 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 03 09:36:32.147080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 03 09:36:32.147166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 03 09:36:32.147259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 03 09:36:32.147337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] command line args: [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 03 09:36:32.147406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] command line args: [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 03 09:36:32.147549 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 03 09:36:32.147640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 03 09:36:32.147718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ================================================================================ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 03 09:36:32.147794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ================================================================================ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 03 09:36:32.147956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] admin_token = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] admin_token = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] config_dir = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] config_dir = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] config_source = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] config_source = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] debug = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] debug = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.148905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.149211 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.149341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.149450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.149555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.149668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.149668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.149861 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150002 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150227 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_config_append = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_config_append = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_dir = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.150962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_dir = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.151071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_file = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.151176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_file = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.151286 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.151416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.151499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.151624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.151705 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.151776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.151902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.152986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153540 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.153930 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.154015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.154093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.154165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.154557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.154641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.154766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] publish_errors = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.154854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] publish_errors = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.154951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.155028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.155175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.155263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.155347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.155472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.155578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.155748 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.155831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.155903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] standard_threads = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156101 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] standard_threads = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156728 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_journal = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_journal = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.156959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_json = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.157030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_json = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.157148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_stderr = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.157229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_stderr = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.157301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] use_syslog = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.157369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] use_syslog = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.157511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.157596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 09:36:32.157761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.157839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.158018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.158098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.158274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.158364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.158510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.158593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.158752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.158835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.158986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.159064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.159228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.159306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.159461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.159539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.159702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.159785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.159943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.160026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.160189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.160273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.160415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.160488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.160649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.160733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.160882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.160968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.161122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.161209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.161366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.161449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.161601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.161679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.161850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.161966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.162128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.162210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.162368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.162453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.162600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.162679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.162850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.162933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.163105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.163189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.163355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.163438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.163601 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.163684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.163830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.163918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.164057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.164132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.164285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.164362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.164516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.164600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.164760 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.164833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.165005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.165070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.165228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.165305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.165530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.165611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.165684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.165756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.165947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.166042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.166185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.166271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.166430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.166509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.166659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.166748 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.167654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.167739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.167889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.167973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.168119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.168204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.168345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.168422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.168572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.168656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.168799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.168887 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.169032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.169123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.169276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.169370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.169504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.169582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.169737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.169816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.169989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.170069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.170222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.170307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.170453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.170538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.170697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.170774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.170923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.171009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.171157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.171240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.171390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.171468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.171620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.171698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.171851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.171936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.172082 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.172161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.172309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.172386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.172552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.172637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.172802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.172917 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.173007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.173081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.173235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.173316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.173489 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.173583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.173721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.173800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.173974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.174055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.174208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.174285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.174440 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.174528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.174702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.174783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.174935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.175013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.175181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.175336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.175491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.175574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.175636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.175746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.175856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.175941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.176088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.176175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.176332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.176409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.176563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.176647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.176796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.176874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.177031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.177115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.177264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.177342 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.177493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.177579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.177726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.177803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.177978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.178061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.178210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.178299 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.178467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.178546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.178703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.178787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.178937 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.179027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.179170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.179248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.179406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.179484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.179640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.179725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.179869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.179946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.180098 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.180176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.180324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.180410 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.180548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.180625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.180773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.180858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.181000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.181148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.181226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.181297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.181447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.181527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.181675 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.181764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.181936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.182016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.182172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.182249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.182470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.183085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.183247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.183327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.183490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.183568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.183722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.external = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.183812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.external = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.183948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.184025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.184239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.184348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.184467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.184552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.184695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.password = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.184781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.password = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.184920 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] auth.token = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.184998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] auth.token = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.185153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.185231 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.185384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.185470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.185624 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.185702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.185850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.185959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.186116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.186208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.186358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.186439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.186585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.186669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.186813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.186962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.187038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.187109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.187287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.187367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.187529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.187616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.187763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.187841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.188000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.188084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.188230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.188321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.188490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.188569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.188741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.188825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.188972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.189058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.189220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.189300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.189496 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.189575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.189738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.189825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.190025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.190105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.190266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.190343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.190517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.190607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.190748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.190828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.190982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.191061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.191225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.191315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.191466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.191545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.191692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.191770 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.191933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.192015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.192168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.192246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.192413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.192491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.192646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.192726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.192896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.192975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.193128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.193207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.193370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.193449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.193600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.193678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.193841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.193976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.194108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.194190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.194343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.194420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.194572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.194650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.194816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.194901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.195041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.195118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.195273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.195358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.195518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.195597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.195763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.195840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.195991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.196070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.196221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.196300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.196447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.196525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.196687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.196766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.196932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.197012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.197174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.197255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.197428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.197516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.197673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.197752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.197962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.198045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.198246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.198391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.198482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.198639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.198718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.198869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.198948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.199104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.199191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.199333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.199413 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.199565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.199645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.199801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.199888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.200037 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.200115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.200276 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.200353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.200503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.200582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.200746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.200831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.200982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.201059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.201211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.201290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.201454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.201533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.201684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.201761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.201932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.202014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.202168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.202246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.202395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.202472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.202631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.202710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.202863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.202941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.203099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.203177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.203333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.203412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.203569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.203649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.203816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.203900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.204049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.204130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.204277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.204354 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.204505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.204591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.204750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.204830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.204983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.205061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.205213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.205289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.205444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.205529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.205672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.205749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.205918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.206002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.206155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.206284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.206520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.206597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.206756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.206833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.207004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.207083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.207250 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.207328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.207484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.207561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.207716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.207796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.207941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.208018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.208174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.208251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.208404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.208483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.208633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.208711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.208860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.208938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.209103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.209183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.209347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.209425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.209590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.209667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.209835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.209939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.210092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.210171 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.210329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.210407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.210564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.210643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.210796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.210874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.211030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.211108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.211262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.211341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.211494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.211572 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.211735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.211814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.211970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.212052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.212201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.212279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.212442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.212519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.212667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.212746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.212908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.212987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.213141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.213218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.213377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.213458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.213639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.213718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.213916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.214000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.214139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.214221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.214368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.214445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.214601 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.214680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.214839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.214919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.215086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.215164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.215323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.215401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.215554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.215633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.215800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.215879 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.216035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.216120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.216266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.216344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.216505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.216582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.216736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.216824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.216967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] role.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.217045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.217196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] role.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.217273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] role.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.217430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.217516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] role.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.217662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.217739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.217934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.218061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.218179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.218288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.218423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.218502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.218663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.218741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.218904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.218992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.219146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.219226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.219389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.219468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.219619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.219706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.219849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.219927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.220092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.220171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.220337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.220426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.220581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.220660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.220824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.220969 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.221207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.221294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.221437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.221515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.221682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.221762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.221949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.222034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.222195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.222275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.222428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.222507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.222658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.222738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.222906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.222985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.223145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.223223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.223377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.223457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.223605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.223684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.223834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.223913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.224062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.224142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.224307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.224387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.224555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.224633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.224794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.224874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.225025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.225103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.225254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.225332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.225479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.225558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.225710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.225789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.225964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.226046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.226210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.226294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.226444 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.226524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.226695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.226773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.226931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.227012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.227160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.227238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.227401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.227480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.227642 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.227722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.227875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.227952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.228104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.228181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.228334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.228413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.228562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.228640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.228802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.228882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.229027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.229106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.229767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.229853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.230040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.230122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.230334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.230423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.230585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.230666 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.230815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.230940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.231053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.231134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.231288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.231376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.231521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.231599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.231750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.231827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.231983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.232070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.232218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.232287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.232463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.232553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.232698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.232787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.232941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.233020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.233176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.233283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.233424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.233512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.233660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.233739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.233923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.234007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.234159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.234251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.234402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.234482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.234636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.234716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.234807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87025) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 03 09:36:32.234895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 09:36:32.235049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87026) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 03 09:36:32.264064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.264239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.264331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.264469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.264556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.264691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.272963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.273056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.301799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.301934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.302053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.302138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.361513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.361833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.410761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.410911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.410999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.411146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.477496 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.477707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.477798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.477954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.478040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.478175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.478271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.478386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.478468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.478596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.478676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.478806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.501906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.502076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.502164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.502306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.502409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.502527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.502611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.502745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.502828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.502959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.503051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.503169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.503250 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.503377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.514463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.514665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.527411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.527638 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.527723 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.527862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.527959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.528082 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.528166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.528297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.528408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.528509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.528601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.528703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87025) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.528781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.528917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87026) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 09:36:32.546037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.546204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.546291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 09:36:32.546411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 09:36:32.546556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.546737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.546827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 09:36:32.546931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 09:36:32.547015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.547193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.547275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.547448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.547537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.547700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.547781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.547950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.559318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.559503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.559606 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.559776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.559859 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.560041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.560124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.560319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.568188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.568369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.571487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.571765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.571967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.572051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.572243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.572325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.572524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.572614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.572802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.572940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.573116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.573234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.573507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.573598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.573776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.573865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.574076 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.574158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.574343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.574457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.574622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.574883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.601017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.601702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.602747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.603027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.603434 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.603707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.608306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.608585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.608862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.609048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.609333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.609610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.616131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.616373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.616999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.617281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.621374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.621768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.622073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.622319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.622403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.622619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.622724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.622894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.623000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.623162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.623274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.623431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.623546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.623813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.624077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.624339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.641990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.642264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.642774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.643048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.646921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.647620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.650580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.650876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.651719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.652008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.656362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.656907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.659990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.660511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.665071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.665376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.665503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.665660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.665800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.665965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.666104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.666250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.666386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.666525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.666661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.667031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.685775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.686081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.686178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.686346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.686456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.686616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.686728 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.686993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.694544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.694868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.696262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.696537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.696620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.696823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.700080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.700335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.700433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.700613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.700695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.700888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.700970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.701160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.701243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.701451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.714632 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.714749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.714873 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.715029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.715142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.715296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.715406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.715560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.715672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.715833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.715944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.716100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.716219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.716367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.734508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.734700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.737743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.737946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.741048 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.741291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.742670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.742908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.744305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.744530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.744614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.744810 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.744921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.745655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.752779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.752988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.754423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.754622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.757588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.757788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.757869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.758097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.758179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.758378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.758472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.758646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.767485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.767670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.767779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.767953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.768159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.768333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.776185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87025) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.776333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87026) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 09:36:32.776461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.776544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.776734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.776815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.777000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.777080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.777267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.777350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.777543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.777622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.777815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.777924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.778113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.778193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.778377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.778456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.778635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.778716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.778898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.778978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.779155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.779238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.779413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.779495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.779673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87025) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.779752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87026) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 09:36:32.818406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557066a389c0 pid: 87025 (default app) Jun 03 09:36:32.818521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557066a389c0 pid: 87026 (default app) Jun 03 09:36:32.821658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afae9f-75c2-473e-ab5e-b7aadefbd33c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:32.821976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afae9f-75c2-473e-ab5e-b7aadefbd33c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:32.822259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afae9f-75c2-473e-ab5e-b7aadefbd33c None None] PATH_INFO: `/v3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:32.826499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Fri Jun 3 09:36:32 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:36:42.005535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c558b7-9340-4862-9014-00637f0c6a21 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:42.006068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c558b7-9340-4862-9014-00637f0c6a21 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:42.006068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c558b7-9340-4862-9014-00637f0c6a21 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:42.010400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:36:42.015618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:42.016028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:42.016182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:42.083577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: INFO dbcounter [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] Registered counter for database keystone Jun 03 09:36:42.105568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG oslo_db.sqlalchemy.engines [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87025) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 03 09:36:42.111693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writer thread running {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 03 09:36:42.388790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG passlib.handlers.bcrypt [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87025) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 03 09:36:42.410858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG passlib.handlers.bcrypt [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87025) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 03 09:36:42.444910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:42.450981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:42.568111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 553 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 03 09:36:42.745436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c558b7-9340-4862-9014-00637f0c6a21 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:42.814469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: INFO dbcounter [None req-90c558b7-9340-4862-9014-00637f0c6a21 None None] Registered counter for database keystone Jun 03 09:36:42.834187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG oslo_db.sqlalchemy.engines [None req-90c558b7-9340-4862-9014-00637f0c6a21 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87026) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 03 09:36:42.840463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writer thread running {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 03 09:36:43.062883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c558b7-9340-4862-9014-00637f0c6a21 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:43.067256 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41679b-6e8b-4b91-927a-b7c1e5c8120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:43.067904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41679b-6e8b-4b91-927a-b7c1e5c8120e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:43.068190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41679b-6e8b-4b91-927a-b7c1e5c8120e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:43.068470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41679b-6e8b-4b91-927a-b7c1e5c8120e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:43.076620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41679b-6e8b-4b91-927a-b7c1e5c8120e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:43.127750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7a41679b-6e8b-4b91-927a-b7c1e5c8120e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 03 09:36:43.128084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41679b-6e8b-4b91-927a-b7c1e5c8120e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:43.137424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Fri Jun 3 09:36:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 03 09:36:43.143445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:43.158070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa5335-acdb-4e53-aa90-897c2593d9a3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:43.162457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12247f-9ce7-4175-984b-69ad18f5831e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:43.163557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12247f-9ce7-4175-984b-69ad18f5831e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:43.163846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12247f-9ce7-4175-984b-69ad18f5831e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:43.164123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12247f-9ce7-4175-984b-69ad18f5831e admin admin] PATH_INFO: `/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:43.165617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12247f-9ce7-4175-984b-69ad18f5831e admin admin] RBAC: Authorizing `identity:get_project(project_id=53d2a61c9fa84503ab8a6c4eb2d3f27e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:43.217732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ab12247f-9ce7-4175-984b-69ad18f5831e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 03 09:36:43.218106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12247f-9ce7-4175-984b-69ad18f5831e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:43.222644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Fri Jun 3 09:36:43 2022] GET /identity/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:45.397622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89b9d2-0b5c-4540-9db9-c8c1c68ff2c5 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:45.398167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89b9d2-0b5c-4540-9db9-c8c1c68ff2c5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:45.398268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89b9d2-0b5c-4540-9db9-c8c1c68ff2c5 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:45.402818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:36:45.408210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68881cc-7d1d-459e-8a97-86bd68711530 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:45.408634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68881cc-7d1d-459e-8a97-86bd68711530 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:45.408775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68881cc-7d1d-459e-8a97-86bd68711530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:45.428198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e68881cc-7d1d-459e-8a97-86bd68711530 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:45.429715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e68881cc-7d1d-459e-8a97-86bd68711530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:45.437587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 03 09:36:45.459737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89b9d2-0b5c-4540-9db9-c8c1c68ff2c5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:45.472297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89b9d2-0b5c-4540-9db9-c8c1c68ff2c5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:45.476439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fa5cb-4ba9-40b5-894a-5f8cd7fe9a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:45.477057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fa5cb-4ba9-40b5-894a-5f8cd7fe9a34 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:45.477340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fa5cb-4ba9-40b5-894a-5f8cd7fe9a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:45.477616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fa5cb-4ba9-40b5-894a-5f8cd7fe9a34 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:45.485213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fa5cb-4ba9-40b5-894a-5f8cd7fe9a34 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:45.486991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fa5cb-4ba9-40b5-894a-5f8cd7fe9a34 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:45.495627 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Fri Jun 3 09:36:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 09:36:45.500826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68881cc-7d1d-459e-8a97-86bd68711530 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:45.509620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68881cc-7d1d-459e-8a97-86bd68711530 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:45.514471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8aa24-09d5-4c18-bdbe-7510b372d0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:45.515323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8aa24-09d5-4c18-bdbe-7510b372d0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:45.515609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8aa24-09d5-4c18-bdbe-7510b372d0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:45.515889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8aa24-09d5-4c18-bdbe-7510b372d0d3 admin admin] PATH_INFO: `/v3/users/e68f65e605c34489879af9763cddb11c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:45.517258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8aa24-09d5-4c18-bdbe-7510b372d0d3 admin admin] RBAC: Authorizing `identity:get_user(user_id=e68f65e605c34489879af9763cddb11c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:45.519131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8aa24-09d5-4c18-bdbe-7510b372d0d3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:45.520668 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Fri Jun 3 09:36:45 2022] GET /identity/v3/users/e68f65e605c34489879af9763cddb11c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:36:47.798784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717ced9-0339-4ddd-b38a-3495305e8211 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:47.799292 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717ced9-0339-4ddd-b38a-3495305e8211 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:47.799364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717ced9-0339-4ddd-b38a-3495305e8211 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:47.800699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:36:47.806063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18435b-8f10-4c2b-9b9b-f5225b2898bd None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:47.806484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18435b-8f10-4c2b-9b9b-f5225b2898bd None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:47.806627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18435b-8f10-4c2b-9b9b-f5225b2898bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:47.823513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8b18435b-8f10-4c2b-9b9b-f5225b2898bd None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:47.824219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f63a8-e81f-4002-8c1e-85de7c9a2914 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:47.824514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f63a8-e81f-4002-8c1e-85de7c9a2914 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:47.824699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8b18435b-8f10-4c2b-9b9b-f5225b2898bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:47.824785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f63a8-e81f-4002-8c1e-85de7c9a2914 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:47.826171 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:36:47.831093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d4894-feb7-470e-a929-b1ffba443307 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:47.831390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d4894-feb7-470e-a929-b1ffba443307 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:47.831674 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d4894-feb7-470e-a929-b1ffba443307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:47.872335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG passlib.handlers.bcrypt [None req-878d4894-feb7-470e-a929-b1ffba443307 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87026) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 03 09:36:47.872774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG passlib.handlers.bcrypt [None req-878d4894-feb7-470e-a929-b1ffba443307 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87026) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 03 09:36:47.901872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:36:47.919501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-878d4894-feb7-470e-a929-b1ffba443307 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:47.921092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-878d4894-feb7-470e-a929-b1ffba443307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:47.924982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18435b-8f10-4c2b-9b9b-f5225b2898bd None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:47.948659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:48.018927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:36:48.028667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18435b-8f10-4c2b-9b9b-f5225b2898bd None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:48.042211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d4894-feb7-470e-a929-b1ffba443307 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:48.062830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65edb9-c8fa-4921-bfcc-b9c6d6297773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:48.063637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65edb9-c8fa-4921-bfcc-b9c6d6297773 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:48.063925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65edb9-c8fa-4921-bfcc-b9c6d6297773 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:48.064204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65edb9-c8fa-4921-bfcc-b9c6d6297773 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:48.075615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65edb9-c8fa-4921-bfcc-b9c6d6297773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-4c65edb9-c8fa-4921-bfcc-b9c6d6297773 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:36:48.086823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:36:48.087652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:36:48.089015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:36:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:36:48.094177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65edb9-c8fa-4921-bfcc-b9c6d6297773 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:48.142196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d4894-feb7-470e-a929-b1ffba443307 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:48.144434 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:48.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:48.179207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010671d1-7c07-47f0-9ec4-7488fd630976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:48.180263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010671d1-7c07-47f0-9ec4-7488fd630976 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:48.180615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010671d1-7c07-47f0-9ec4-7488fd630976 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:48.180962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010671d1-7c07-47f0-9ec4-7488fd630976 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:48.192407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65edb9-c8fa-4921-bfcc-b9c6d6297773 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:48.226004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a71cb-c654-4cf4-970d-367216aa81ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:48.226665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a71cb-c654-4cf4-970d-367216aa81ff None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:48.226954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a71cb-c654-4cf4-970d-367216aa81ff None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:48.227235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a71cb-c654-4cf4-970d-367216aa81ff None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:48.227987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a71cb-c654-4cf4-970d-367216aa81ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:36:48.228294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a71cb-c654-4cf4-970d-367216aa81ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:48.229704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a71cb-c654-4cf4-970d-367216aa81ff None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:48.242408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:36:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:48.248382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a71cb-c654-4cf4-970d-367216aa81ff None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:48.251660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010671d1-7c07-47f0-9ec4-7488fd630976 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:48.253173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010671d1-7c07-47f0-9ec4-7488fd630976 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:48.343979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a71cb-c654-4cf4-970d-367216aa81ff None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:48.347757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:36:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 09:36:48.353135 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010671d1-7c07-47f0-9ec4-7488fd630976 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:48.355842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:48.377668 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb457c-311c-4b34-9112-ae2ff7775c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:48.378568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb457c-311c-4b34-9112-ae2ff7775c30 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:48.378852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb457c-311c-4b34-9112-ae2ff7775c30 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:48.379128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb457c-311c-4b34-9112-ae2ff7775c30 None admin] PATH_INFO: `/v3/users/e68f65e605c34489879af9763cddb11c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:48.380418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb457c-311c-4b34-9112-ae2ff7775c30 None admin] RBAC: Authorizing `identity:get_user(user_id=e68f65e605c34489879af9763cddb11c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:48.381897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb457c-311c-4b34-9112-ae2ff7775c30 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:48.383505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:36:48 2022] GET /identity/v3/users/e68f65e605c34489879af9763cddb11c => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:48.389301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb457c-311c-4b34-9112-ae2ff7775c30 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:48.403595 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:48.452229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010671d1-7c07-47f0-9ec4-7488fd630976 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:48.485845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1cf3f-0238-453b-a7e7-dc5b617c273d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:48.486558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb457c-311c-4b34-9112-ae2ff7775c30 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:48.486664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1cf3f-0238-453b-a7e7-dc5b617c273d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:48.486940 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1cf3f-0238-453b-a7e7-dc5b617c273d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:48.487224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1cf3f-0238-453b-a7e7-dc5b617c273d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:48.505043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1cf3f-0238-453b-a7e7-dc5b617c273d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:48.506551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1cf3f-0238-453b-a7e7-dc5b617c273d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:48.520102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b9ae8-15fa-4f96-939b-b9ec96c4e3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:48.520941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b9ae8-15fa-4f96-939b-b9ec96c4e3e6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:48.521233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b9ae8-15fa-4f96-939b-b9ec96c4e3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:48.521515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b9ae8-15fa-4f96-939b-b9ec96c4e3e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:48.522725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b9ae8-15fa-4f96-939b-b9ec96c4e3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:48.524171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b9ae8-15fa-4f96-939b-b9ec96c4e3e6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:48.525842 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:36:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-91e1cf3f-0238-453b-a7e7-dc5b617c273d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:36:48.527346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:36:48.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:36:48.529094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:36:48.529094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:36:48.529094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:36:48.529094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:36:48.529094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:36:48.531416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b9ae8-15fa-4f96-939b-b9ec96c4e3e6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:48.534429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1cf3f-0238-453b-a7e7-dc5b617c273d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:48.590775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:48.611293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:48.629502 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b9ae8-15fa-4f96-939b-b9ec96c4e3e6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:48.633794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1cf3f-0238-453b-a7e7-dc5b617c273d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:48.664173 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd545d5-5611-4150-857d-71068b28e344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:48.664823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd545d5-5611-4150-857d-71068b28e344 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:48.665109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd545d5-5611-4150-857d-71068b28e344 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:48.665388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd545d5-5611-4150-857d-71068b28e344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:48.666272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd545d5-5611-4150-857d-71068b28e344 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.domain.id=default, user.id=e68f65e605c34489879af9763cddb11c) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:36:48.666581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd545d5-5611-4150-857d-71068b28e344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:48.667617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2c056-96a6-4131-b9a7-5822a361f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:48.668271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c056-96a6-4131-b9a7-5822a361f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:48.668355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd545d5-5611-4150-857d-71068b28e344 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:48.668558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c056-96a6-4131-b9a7-5822a361f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:48.668837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c056-96a6-4131-b9a7-5822a361f1a7 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:48.669554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2c056-96a6-4131-b9a7-5822a361f1a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:36:48.669862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2c056-96a6-4131-b9a7-5822a361f1a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:48.671289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2c056-96a6-4131-b9a7-5822a361f1a7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:48.675336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Fri Jun 3 09:36:48 2022] GET /identity/v3/role_assignments?user.id=e68f65e605c34489879af9763cddb11c&scope.domain.id=default&role.id=fe4d636f5c4842f18569347911e6cdec => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:36:48.689960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:51.263656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854152fa-fa5e-41e5-aa98-095a8c70776d None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:51.264165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854152fa-fa5e-41e5-aa98-095a8c70776d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:51.264238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854152fa-fa5e-41e5-aa98-095a8c70776d None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:51.265575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:36:51.270857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a7cb8-fa16-4dbf-92ba-4ddbfefee7ae None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:51.271237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a7cb8-fa16-4dbf-92ba-4ddbfefee7ae None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:51.271428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a7cb8-fa16-4dbf-92ba-4ddbfefee7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:51.298874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-346a7cb8-fa16-4dbf-92ba-4ddbfefee7ae None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:51.300075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-346a7cb8-fa16-4dbf-92ba-4ddbfefee7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:51.371151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d3873-f4be-4ca0-a24e-33d7eaf94abf None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:51.371458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d3873-f4be-4ca0-a24e-33d7eaf94abf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:51.371723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d3873-f4be-4ca0-a24e-33d7eaf94abf None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:51.373081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 13/23] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:36:51.379604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21aa3cb-c5e0-4121-bd17-078f45185b3d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:51.379896 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21aa3cb-c5e0-4121-bd17-078f45185b3d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:51.380337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21aa3cb-c5e0-4121-bd17-078f45185b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:51.391883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 11/24] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:36:51.410915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d21aa3cb-c5e0-4121-bd17-078f45185b3d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:51.412150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d21aa3cb-c5e0-4121-bd17-078f45185b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:51.434662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a7cb8-fa16-4dbf-92ba-4ddbfefee7ae None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:51.523623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:51.526379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 14/25] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:36:51.551204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb755c-d446-4d82-a43d-0e944494ba3b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:51.551888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb755c-d446-4d82-a43d-0e944494ba3b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:51.552410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb755c-d446-4d82-a43d-0e944494ba3b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:51.553995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 15/26] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:36:51.559686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f91b4-051f-43b3-aa58-d4ca8c08fdd7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:51.560306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f91b4-051f-43b3-aa58-d4ca8c08fdd7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:51.560784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f91b4-051f-43b3-aa58-d4ca8c08fdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:51.595652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-153f91b4-051f-43b3-aa58-d4ca8c08fdd7 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:51.598052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-153f91b4-051f-43b3-aa58-d4ca8c08fdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:51.623624 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a7cb8-fa16-4dbf-92ba-4ddbfefee7ae None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:51.691966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3db40-a9c4-4925-a0d4-f531fddebfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:51.692770 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3db40-a9c4-4925-a0d4-f531fddebfe3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:51.693054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3db40-a9c4-4925-a0d4-f531fddebfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:51.693332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3db40-a9c4-4925-a0d4-f531fddebfe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:51.714942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3db40-a9c4-4925-a0d4-f531fddebfe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:51.716435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3db40-a9c4-4925-a0d4-f531fddebfe3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:51.750305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 16/27] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:36:51.751300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f91b4-051f-43b3-aa58-d4ca8c08fdd7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2df3db40-a9c4-4925-a0d4-f531fddebfe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:36:51.759605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:36:51.760958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:36:51.761814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:36:51.761814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:36:51.761814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:36:51.761814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:36:51.763846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:51.764323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 12/28] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:36:51.786714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3db40-a9c4-4925-a0d4-f531fddebfe3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:51.867132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f91b4-051f-43b3-aa58-d4ca8c08fdd7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:51.913956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d723466-b5b0-4998-b14a-1c1a05d364b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:51.915178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d723466-b5b0-4998-b14a-1c1a05d364b5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:51.915782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d723466-b5b0-4998-b14a-1c1a05d364b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:51.916299 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d723466-b5b0-4998-b14a-1c1a05d364b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:51.937752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d723466-b5b0-4998-b14a-1c1a05d364b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3d723466-b5b0-4998-b14a-1c1a05d364b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:36:51.947987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:36:51.948989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:36:51.949799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:36:51.951316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 17/29] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:36:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:36:51.952803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d723466-b5b0-4998-b14a-1c1a05d364b5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:52.019625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:52.024020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3db40-a9c4-4925-a0d4-f531fddebfe3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:52.061812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab52b38-972c-439a-afbf-2ed31def2708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:52.062648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab52b38-972c-439a-afbf-2ed31def2708 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.062935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab52b38-972c-439a-afbf-2ed31def2708 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.063461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab52b38-972c-439a-afbf-2ed31def2708 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.064194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab52b38-972c-439a-afbf-2ed31def2708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:36:52.064506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab52b38-972c-439a-afbf-2ed31def2708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:52.065995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab52b38-972c-439a-afbf-2ed31def2708 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:52.074232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:52.080686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 13/30] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:52.081653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab52b38-972c-439a-afbf-2ed31def2708 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:52.151905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d723466-b5b0-4998-b14a-1c1a05d364b5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:52.221956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab52b38-972c-439a-afbf-2ed31def2708 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:52.233040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb480074-d935-4d76-a6ae-39138a33cdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:52.233730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb480074-d935-4d76-a6ae-39138a33cdfa None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.234054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb480074-d935-4d76-a6ae-39138a33cdfa None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.234340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb480074-d935-4d76-a6ae-39138a33cdfa None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.235089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb480074-d935-4d76-a6ae-39138a33cdfa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:52.236495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb480074-d935-4d76-a6ae-39138a33cdfa None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:52.261230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f7ba0-2084-4566-95d5-fccbaaf6cb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:52.268185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f7ba0-2084-4566-95d5-fccbaaf6cb1e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.268185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f7ba0-2084-4566-95d5-fccbaaf6cb1e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.268185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f7ba0-2084-4566-95d5-fccbaaf6cb1e None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.268185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f7ba0-2084-4566-95d5-fccbaaf6cb1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:36:52.268185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f7ba0-2084-4566-95d5-fccbaaf6cb1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:52.268185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f7ba0-2084-4566-95d5-fccbaaf6cb1e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:52.278252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 14/31] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:36:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:36:52.279075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f7ba0-2084-4566-95d5-fccbaaf6cb1e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:52.303418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:52.393632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f7ba0-2084-4566-95d5-fccbaaf6cb1e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:52.437379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cc6c0-5cc0-4f14-8368-aecf38fc4a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:52.438554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cc6c0-5cc0-4f14-8368-aecf38fc4a5f None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.439010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cc6c0-5cc0-4f14-8368-aecf38fc4a5f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.439487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cc6c0-5cc0-4f14-8368-aecf38fc4a5f None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.440689 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cc6c0-5cc0-4f14-8368-aecf38fc4a5f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:52.442550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cc6c0-5cc0-4f14-8368-aecf38fc4a5f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:52.501086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 18/32] 10.222.0.42 () {60 vars in 1131 bytes} [Fri Jun 3 09:36:51 2022] POST /identity/v3/roles => generated 219 bytes in 549 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:36:52.509338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb480074-d935-4d76-a6ae-39138a33cdfa None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:52.711159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb480074-d935-4d76-a6ae-39138a33cdfa None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:52.719891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 15/33] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:36:52 2022] POST /identity/v3/projects => generated 301 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:36:52.719891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dc291-03fd-42ce-9cef-e4931a2e6f19 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.719891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dc291-03fd-42ce-9cef-e4931a2e6f19 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.719891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855dc291-03fd-42ce-9cef-e4931a2e6f19 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.719891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 16/34] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:36:52.729924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c888b7e-413a-4d22-b436-ed0733cf617a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.729924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c888b7e-413a-4d22-b436-ed0733cf617a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.729924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c888b7e-413a-4d22-b436-ed0733cf617a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.768071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5c888b7e-413a-4d22-b436-ed0733cf617a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:52.769634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5c888b7e-413a-4d22-b436-ed0733cf617a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:52.770266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:52.788637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb72a3a-3387-4f46-8a97-637484eb50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:52.789533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb72a3a-3387-4f46-8a97-637484eb50a6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.789813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb72a3a-3387-4f46-8a97-637484eb50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.790116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb72a3a-3387-4f46-8a97-637484eb50a6 None admin] PATH_INFO: `/v3/users/e68f65e605c34489879af9763cddb11c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.793734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb72a3a-3387-4f46-8a97-637484eb50a6 None admin] RBAC: Authorizing `identity:get_user(user_id=e68f65e605c34489879af9763cddb11c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:52.795374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb72a3a-3387-4f46-8a97-637484eb50a6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:52.798053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 19/35] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:36:52 2022] GET /identity/v3/users/e68f65e605c34489879af9763cddb11c => generated 247 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:36:52.800608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9b001-b381-49c8-9c69-70fd67063032 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.801109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9b001-b381-49c8-9c69-70fd67063032 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.801562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c9b001-b381-49c8-9c69-70fd67063032 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.803305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 20/36] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:36:52.807061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9b001-b381-49c8-9c69-70fd67063032 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:52.878333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 17/37] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:36:52.887458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff9d03-8e3a-41fe-95d7-4a350442844f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.887754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff9d03-8e3a-41fe-95d7-4a350442844f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.888032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff9d03-8e3a-41fe-95d7-4a350442844f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.898535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:52.913310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0eff9d03-8e3a-41fe-95d7-4a350442844f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:52.914649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0eff9d03-8e3a-41fe-95d7-4a350442844f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:52.945600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c9b001-b381-49c8-9c69-70fd67063032 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:52.987677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c000a-ccd1-436f-aadb-ebb8f19e7417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:52.988833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c000a-ccd1-436f-aadb-ebb8f19e7417 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.989123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c000a-ccd1-436f-aadb-ebb8f19e7417 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.989406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c000a-ccd1-436f-aadb-ebb8f19e7417 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.991764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c000a-ccd1-436f-aadb-ebb8f19e7417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:52.993255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c000a-ccd1-436f-aadb-ebb8f19e7417 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:52.995109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 21/38] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:36:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:36:52.995795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 18/39] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:36:52.996915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8616b-eee6-42c0-9ffb-63a332adcdbc None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:52.997204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8616b-eee6-42c0-9ffb-63a332adcdbc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:52.997482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8616b-eee6-42c0-9ffb-63a332adcdbc None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:52.998952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 19/40] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:36:52.999226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c000a-ccd1-436f-aadb-ebb8f19e7417 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:53.003542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8616b-eee6-42c0-9ffb-63a332adcdbc None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:53.071181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:53.113950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c000a-ccd1-436f-aadb-ebb8f19e7417 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:53.116552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8616b-eee6-42c0-9ffb-63a332adcdbc None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:53.146581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:53.146793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb3927-ad35-453f-b536-50fc5bfa7f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:53.147499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb3927-ad35-453f-b536-50fc5bfa7f51 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:53.147813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb3927-ad35-453f-b536-50fc5bfa7f51 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:53.148002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb3927-ad35-453f-b536-50fc5bfa7f51 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:53.148735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb3927-ad35-453f-b536-50fc5bfa7f51 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:53.150146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb3927-ad35-453f-b536-50fc5bfa7f51 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:53.152999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f463b6-002c-4524-a171-b6740722084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:53.154474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f463b6-002c-4524-a171-b6740722084c None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:53.154772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f463b6-002c-4524-a171-b6740722084c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:53.155102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f463b6-002c-4524-a171-b6740722084c None admin] PATH_INFO: `/v3/domains/default/users/e68f65e605c34489879af9763cddb11c/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:53.157431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f463b6-002c-4524-a171-b6740722084c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e68f65e605c34489879af9763cddb11c, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:53.159045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f463b6-002c-4524-a171-b6740722084c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:53.238901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 22/41] 10.222.0.42 () {60 vars in 1131 bytes} [Fri Jun 3 09:36:52 2022] POST /identity/v3/roles => generated 225 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:36:53.240074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8be47-810c-4c6d-a79c-d00e20f39c37 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:53.240539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8be47-810c-4c6d-a79c-d00e20f39c37 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:53.241008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8be47-810c-4c6d-a79c-d00e20f39c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:53.251106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 20/42] 10.222.0.42 () {58 vars in 1361 bytes} [Fri Jun 3 09:36:53 2022] PUT /identity/v3/domains/default/users/e68f65e605c34489879af9763cddb11c/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:36:53.252286 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f463b6-002c-4524-a171-b6740722084c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:53.269060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cda8be47-810c-4c6d-a79c-d00e20f39c37 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:53.274887 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cda8be47-810c-4c6d-a79c-d00e20f39c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:53.351335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 23/43] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:36:53.376260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8be47-810c-4c6d-a79c-d00e20f39c37 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:53.409826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f463b6-002c-4524-a171-b6740722084c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:53.449115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:53.450823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98977db-1757-4339-8ffa-f3b286780e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:53.451695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98977db-1757-4339-8ffa-f3b286780e34 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:53.451997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98977db-1757-4339-8ffa-f3b286780e34 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:53.452282 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98977db-1757-4339-8ffa-f3b286780e34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:53.454247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98977db-1757-4339-8ffa-f3b286780e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:53.455857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98977db-1757-4339-8ffa-f3b286780e34 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:53.457632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 21/44] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:36:53.464363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98977db-1757-4339-8ffa-f3b286780e34 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:53.519598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8be47-810c-4c6d-a79c-d00e20f39c37 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:53.540167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:53.566466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfae48-761e-4559-9bfc-0a9caa1f8210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:53.567152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfae48-761e-4559-9bfc-0a9caa1f8210 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:53.567442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfae48-761e-4559-9bfc-0a9caa1f8210 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:53.567882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfae48-761e-4559-9bfc-0a9caa1f8210 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:53.569055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfae48-761e-4559-9bfc-0a9caa1f8210 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:53.570785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfae48-761e-4559-9bfc-0a9caa1f8210 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:53.586817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98977db-1757-4339-8ffa-f3b286780e34 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:53.623088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a880973-e89f-42c2-babf-a078ef16e7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:53.624115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a880973-e89f-42c2-babf-a078ef16e7b6 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:53.624550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a880973-e89f-42c2-babf-a078ef16e7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:53.624935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a880973-e89f-42c2-babf-a078ef16e7b6 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:53.626111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a880973-e89f-42c2-babf-a078ef16e7b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:53.628550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a880973-e89f-42c2-babf-a078ef16e7b6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:53.662505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 24/45] 10.222.0.42 () {60 vars in 1131 bytes} [Fri Jun 3 09:36:53 2022] POST /identity/v3/roles => generated 223 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:36:53.715183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 22/46] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:36:53 2022] POST /identity/v3/projects => generated 312 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:36:55.567426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7422a9-4859-4f54-b7e8-7eab217d02f9 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:55.567965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7422a9-4859-4f54-b7e8-7eab217d02f9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:55.567965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7422a9-4859-4f54-b7e8-7eab217d02f9 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:55.569344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 25/47] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:36:55.574643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f668-a5cf-458b-8752-a7e22a1bd120 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:55.575152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f668-a5cf-458b-8752-a7e22a1bd120 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:55.575247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f668-a5cf-458b-8752-a7e22a1bd120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:55.591941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d822f668-a5cf-458b-8752-a7e22a1bd120 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:55.593120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d822f668-a5cf-458b-8752-a7e22a1bd120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:55.661133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 23/48] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:36:55.685200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7422a9-4859-4f54-b7e8-7eab217d02f9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:55.788043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7422a9-4859-4f54-b7e8-7eab217d02f9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:55.798901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:55.822730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b992f8-6333-49cd-9e86-8b63970523a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:55.823476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b992f8-6333-49cd-9e86-8b63970523a4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:55.823768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b992f8-6333-49cd-9e86-8b63970523a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:55.824052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b992f8-6333-49cd-9e86-8b63970523a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:55.835160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b992f8-6333-49cd-9e86-8b63970523a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a1b992f8-6333-49cd-9e86-8b63970523a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:36:55.843682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:36:55.844727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:36:55.845652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 26/49] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:36:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:36:55.852419 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822f668-a5cf-458b-8752-a7e22a1bd120 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:55.894799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:55.951919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822f668-a5cf-458b-8752-a7e22a1bd120 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:55.982448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a121a72-cce5-4696-b6dd-c1cc66ad7fa9 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:55.982734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a121a72-cce5-4696-b6dd-c1cc66ad7fa9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:55.983024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a121a72-cce5-4696-b6dd-c1cc66ad7fa9 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:55.984394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 27/50] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:36:55.985980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffd958-9a69-43e7-b962-f7ceda4da561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:55.986657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffd958-9a69-43e7-b962-f7ceda4da561 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:55.986946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffd958-9a69-43e7-b962-f7ceda4da561 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:55.987233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffd958-9a69-43e7-b962-f7ceda4da561 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:55.987990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffd958-9a69-43e7-b962-f7ceda4da561 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:36:55.988310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffd958-9a69-43e7-b962-f7ceda4da561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:55.989327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9382004-1109-4516-8e6c-5ec9826a3dbd None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:55.989611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9382004-1109-4516-8e6c-5ec9826a3dbd None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:55.989699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bffd958-9a69-43e7-b962-f7ceda4da561 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:55.989908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9382004-1109-4516-8e6c-5ec9826a3dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:55.998067 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 24/51] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:36:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:36:56.003496 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffd958-9a69-43e7-b962-f7ceda4da561 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:56.006373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c9382004-1109-4516-8e6c-5ec9826a3dbd None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:56.007547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c9382004-1109-4516-8e6c-5ec9826a3dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:56.074895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 28/52] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:36:56.097478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9382004-1109-4516-8e6c-5ec9826a3dbd None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:56.099921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bffd958-9a69-43e7-b962-f7ceda4da561 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:56.102228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:56.135117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d93e2-7988-4d6d-b1b0-e64c9ea1fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:56.135998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d93e2-7988-4d6d-b1b0-e64c9ea1fa96 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:56.136282 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d93e2-7988-4d6d-b1b0-e64c9ea1fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:56.136561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d93e2-7988-4d6d-b1b0-e64c9ea1fa96 None admin] PATH_INFO: `/v3/users/e68f65e605c34489879af9763cddb11c` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:56.137759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d93e2-7988-4d6d-b1b0-e64c9ea1fa96 None admin] RBAC: Authorizing `identity:get_user(user_id=e68f65e605c34489879af9763cddb11c)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:56.139246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d93e2-7988-4d6d-b1b0-e64c9ea1fa96 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:56.141018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 25/53] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:36:56 2022] GET /identity/v3/users/e68f65e605c34489879af9763cddb11c => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:56.146254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d93e2-7988-4d6d-b1b0-e64c9ea1fa96 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:56.191780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:56.195080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9382004-1109-4516-8e6c-5ec9826a3dbd None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:56.228903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcea8b-e980-4148-bdc9-1310ad7c6dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:56.229686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcea8b-e980-4148-bdc9-1310ad7c6dc0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:56.230005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcea8b-e980-4148-bdc9-1310ad7c6dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:56.230278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcea8b-e980-4148-bdc9-1310ad7c6dc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:56.231444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcea8b-e980-4148-bdc9-1310ad7c6dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:56.232899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcea8b-e980-4148-bdc9-1310ad7c6dc0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:56.234523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 29/54] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:56.239837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcea8b-e980-4148-bdc9-1310ad7c6dc0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:56.246475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d93e2-7988-4d6d-b1b0-e64c9ea1fa96 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:56.280675 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a6629-cd96-4e99-98f7-c9d720ee472f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:56.281495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a6629-cd96-4e99-98f7-c9d720ee472f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:56.281788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a6629-cd96-4e99-98f7-c9d720ee472f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:56.282103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a6629-cd96-4e99-98f7-c9d720ee472f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:56.283293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a6629-cd96-4e99-98f7-c9d720ee472f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:56.284737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a6629-cd96-4e99-98f7-c9d720ee472f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:56.286377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 26/55] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:56.292945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a6629-cd96-4e99-98f7-c9d720ee472f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:56.333321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:56.337042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcea8b-e980-4148-bdc9-1310ad7c6dc0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:56.370438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d8906-1d11-4989-891d-4a592ca343e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:56.371107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d8906-1d11-4989-891d-4a592ca343e8 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:56.371397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d8906-1d11-4989-891d-4a592ca343e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:56.371680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d8906-1d11-4989-891d-4a592ca343e8 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:56.372390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d8906-1d11-4989-891d-4a592ca343e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:56.373838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d8906-1d11-4989-891d-4a592ca343e8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:56.390508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a6629-cd96-4e99-98f7-c9d720ee472f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:56.423045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9789f-1dce-4f88-9d45-4d201ecbb0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:56.423692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9789f-1dce-4f88-9d45-4d201ecbb0b0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:56.423980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9789f-1dce-4f88-9d45-4d201ecbb0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:56.424271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9789f-1dce-4f88-9d45-4d201ecbb0b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:56.425099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9789f-1dce-4f88-9d45-4d201ecbb0b0 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.domain.id=default, user.id=e68f65e605c34489879af9763cddb11c) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:36:56.425415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9789f-1dce-4f88-9d45-4d201ecbb0b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:56.426878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9789f-1dce-4f88-9d45-4d201ecbb0b0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:56.435335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 27/56] 10.222.0.42 () {56 vars in 1322 bytes} [Fri Jun 3 09:36:56 2022] GET /identity/v3/role_assignments?user.id=e68f65e605c34489879af9763cddb11c&scope.domain.id=default&role.id=fe4d636f5c4842f18569347911e6cdec => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:36:56.453205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 30/57] 10.222.0.42 () {60 vars in 1140 bytes} [Fri Jun 3 09:36:56 2022] POST /identity/v3/projects => generated 298 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:36:58.763200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8945b10-4733-4900-92e0-0f0ebc7cd42b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:58.763629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8945b10-4733-4900-92e0-0f0ebc7cd42b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:58.763776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8945b10-4733-4900-92e0-0f0ebc7cd42b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:58.765158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 28/58] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:36:58.770359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6a75e-c1a9-4186-80af-cb077de2ccc1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:58.770762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6a75e-c1a9-4186-80af-cb077de2ccc1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:58.770916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6a75e-c1a9-4186-80af-cb077de2ccc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:58.787662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9de6a75e-c1a9-4186-80af-cb077de2ccc1 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:36:58.788837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9de6a75e-c1a9-4186-80af-cb077de2ccc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:36:58.803089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:58.853983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 31/59] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:36:58.878140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8945b10-4733-4900-92e0-0f0ebc7cd42b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:58.977653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8945b10-4733-4900-92e0-0f0ebc7cd42b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:58.979877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:36:59.009295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b91dc-4a94-45b4-ae98-86b20053217f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:59.010804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b91dc-4a94-45b4-ae98-86b20053217f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:59.011307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b91dc-4a94-45b4-ae98-86b20053217f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:59.011649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b91dc-4a94-45b4-ae98-86b20053217f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:59.013114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b91dc-4a94-45b4-ae98-86b20053217f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:59.014628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b91dc-4a94-45b4-ae98-86b20053217f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:59.016272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 29/60] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:36:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:36:59.021791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6a75e-c1a9-4186-80af-cb077de2ccc1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:36:59.118481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6a75e-c1a9-4186-80af-cb077de2ccc1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:36:59.151605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62181380-e733-47d2-8759-072c24ddf22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:36:59.152274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62181380-e733-47d2-8759-072c24ddf22c None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:36:59.152562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62181380-e733-47d2-8759-072c24ddf22c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:36:59.152845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62181380-e733-47d2-8759-072c24ddf22c None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:36:59.153565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62181380-e733-47d2-8759-072c24ddf22c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:36:59.155050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62181380-e733-47d2-8759-072c24ddf22c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:36:59.266713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 32/61] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:36:59 2022] POST /identity/v3/users => generated 275 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:37:02.665709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9e98-fd52-43ed-a25b-cc0c5b091bdd None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:02.666999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9e98-fd52-43ed-a25b-cc0c5b091bdd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:02.667442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9e98-fd52-43ed-a25b-cc0c5b091bdd None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:02.669192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 30/62] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:02.674823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e875e4-f8ad-4a3b-8aca-06c0787c562d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:02.675595 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e875e4-f8ad-4a3b-8aca-06c0787c562d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:02.676026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e875e4-f8ad-4a3b-8aca-06c0787c562d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:02.688153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:02.700182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a7e875e4-f8ad-4a3b-8aca-06c0787c562d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:02.701394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a7e875e4-f8ad-4a3b-8aca-06c0787c562d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:02.793244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 33/63] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:02.823146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9e98-fd52-43ed-a25b-cc0c5b091bdd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:02.829591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be19f3-41e4-4868-bf94-9c35bcfd0504 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:02.830160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be19f3-41e4-4868-bf94-9c35bcfd0504 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:02.830694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be19f3-41e4-4868-bf94-9c35bcfd0504 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:02.832254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 34/64] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:02.837445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d843d-75e3-4e96-ab46-70d0082724de None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:02.838089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d843d-75e3-4e96-ab46-70d0082724de None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:02.838581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d843d-75e3-4e96-ab46-70d0082724de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:02.867425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-350d843d-75e3-4e96-ab46-70d0082724de None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:02.871417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-350d843d-75e3-4e96-ab46-70d0082724de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:02.943070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 35/65] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:02.944530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25587d-a4bc-44ba-964f-4c94bb5be229 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:02.944973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25587d-a4bc-44ba-964f-4c94bb5be229 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:02.945395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25587d-a4bc-44ba-964f-4c94bb5be229 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:02.947375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 36/66] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:02.955151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683d8d4-b6e8-47ee-9aa4-2804f22f5131 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:02.955626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683d8d4-b6e8-47ee-9aa4-2804f22f5131 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:02.956051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683d8d4-b6e8-47ee-9aa4-2804f22f5131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:02.970185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9e98-fd52-43ed-a25b-cc0c5b091bdd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:02.980852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b683d8d4-b6e8-47ee-9aa4-2804f22f5131 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:02.982540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b683d8d4-b6e8-47ee-9aa4-2804f22f5131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:03.007801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da1c0-24fb-47cb-9fa3-00ea299e2074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.009081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da1c0-24fb-47cb-9fa3-00ea299e2074 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.010154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da1c0-24fb-47cb-9fa3-00ea299e2074 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.010661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da1c0-24fb-47cb-9fa3-00ea299e2074 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.025007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5da1c0-24fb-47cb-9fa3-00ea299e2074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ef5da1c0-24fb-47cb-9fa3-00ea299e2074 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:03.034923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:03.035828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:03.036578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:03.044575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 31/67] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:37:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:03.046019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da1c0-24fb-47cb-9fa3-00ea299e2074 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.059117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:03.070318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 37/68] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:03.071367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683d8d4-b6e8-47ee-9aa4-2804f22f5131 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.098469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:03.178020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da1c0-24fb-47cb-9fa3-00ea299e2074 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.184444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683d8d4-b6e8-47ee-9aa4-2804f22f5131 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.212153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474aa63-bbc7-44c1-bd1b-2bd81e01515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.213433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474aa63-bbc7-44c1-bd1b-2bd81e01515d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.213725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474aa63-bbc7-44c1-bd1b-2bd81e01515d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.214041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474aa63-bbc7-44c1-bd1b-2bd81e01515d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.219887 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91c1f-0d86-457a-85fc-246b2f5940c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.220660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91c1f-0d86-457a-85fc-246b2f5940c0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.220942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91c1f-0d86-457a-85fc-246b2f5940c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.221394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91c1f-0d86-457a-85fc-246b2f5940c0 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.222360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91c1f-0d86-457a-85fc-246b2f5940c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:03.222806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91c1f-0d86-457a-85fc-246b2f5940c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:03.224254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474aa63-bbc7-44c1-bd1b-2bd81e01515d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:03.224476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91c1f-0d86-457a-85fc-246b2f5940c0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d474aa63-bbc7-44c1-bd1b-2bd81e01515d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:03.232499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:03.233483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:03.234300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 32/69] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:03.234894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474aa63-bbc7-44c1-bd1b-2bd81e01515d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.235513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 38/70] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:03.236886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d519020-b0c5-404d-af3b-5b02914c9489 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.237307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d519020-b0c5-404d-af3b-5b02914c9489 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.237702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d519020-b0c5-404d-af3b-5b02914c9489 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.239351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 39/71] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:03.240165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d519020-b0c5-404d-af3b-5b02914c9489 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.305364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:03.337017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:03.346078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474aa63-bbc7-44c1-bd1b-2bd81e01515d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.349523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d519020-b0c5-404d-af3b-5b02914c9489 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.380247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f021f7-ddd5-409f-8917-a5cef657d4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.381053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f021f7-ddd5-409f-8917-a5cef657d4d8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.381348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f021f7-ddd5-409f-8917-a5cef657d4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.381627 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f021f7-ddd5-409f-8917-a5cef657d4d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.382805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19aa71-3c9b-4ad9-9d0a-28d70dbba884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.383471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19aa71-3c9b-4ad9-9d0a-28d70dbba884 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.383757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19aa71-3c9b-4ad9-9d0a-28d70dbba884 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.384297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19aa71-3c9b-4ad9-9d0a-28d70dbba884 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.385391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19aa71-3c9b-4ad9-9d0a-28d70dbba884 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:03.385697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19aa71-3c9b-4ad9-9d0a-28d70dbba884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:03.387103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19aa71-3c9b-4ad9-9d0a-28d70dbba884 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:03.392044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f021f7-ddd5-409f-8917-a5cef657d4d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:03.395008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 40/72] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:03.395801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19aa71-3c9b-4ad9-9d0a-28d70dbba884 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-f8f021f7-ddd5-409f-8917-a5cef657d4d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:03.400164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:03.401018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:03.401752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 33/73] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:03.402850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327f917-cdf2-472d-b6f3-d19a4e929fde None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.402850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327f917-cdf2-472d-b6f3-d19a4e929fde None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.403104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327f917-cdf2-472d-b6f3-d19a4e929fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.419926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b327f917-cdf2-472d-b6f3-d19a4e929fde None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:03.421109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b327f917-cdf2-472d-b6f3-d19a4e929fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:03.484594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 34/74] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:03.487226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7472eb-e51e-4570-8e88-6cf9aaba2093 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.487524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7472eb-e51e-4570-8e88-6cf9aaba2093 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.487815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7472eb-e51e-4570-8e88-6cf9aaba2093 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.489111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 35/75] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:03.489813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7472eb-e51e-4570-8e88-6cf9aaba2093 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.498290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19aa71-3c9b-4ad9-9d0a-28d70dbba884 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.518723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:03.532587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22c00d-b5d4-477a-a3e4-a2ac7ea91ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.535124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c00d-b5d4-477a-a3e4-a2ac7ea91ab5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.535124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c00d-b5d4-477a-a3e4-a2ac7ea91ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.535124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c00d-b5d4-477a-a3e4-a2ac7ea91ab5 None admin] PATH_INFO: `/v3/users/e68f65e605c34489879af9763cddb11c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.535363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22c00d-b5d4-477a-a3e4-a2ac7ea91ab5 None admin] RBAC: Authorizing `identity:get_user(user_id=e68f65e605c34489879af9763cddb11c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:03.536669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22c00d-b5d4-477a-a3e4-a2ac7ea91ab5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:03.538477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 41/76] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/users/e68f65e605c34489879af9763cddb11c => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:03.539159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22c00d-b5d4-477a-a3e4-a2ac7ea91ab5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.573714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:03.591820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7472eb-e51e-4570-8e88-6cf9aaba2093 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.625365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268240c-e222-464d-91f0-789308bab3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.626266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268240c-e222-464d-91f0-789308bab3c9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.626548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268240c-e222-464d-91f0-789308bab3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.626834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268240c-e222-464d-91f0-789308bab3c9 None admin] PATH_INFO: `/v3/users/c987d6e5675d41e7b4123050fafb4d99` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.638866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22c00d-b5d4-477a-a3e4-a2ac7ea91ab5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.641840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268240c-e222-464d-91f0-789308bab3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=c987d6e5675d41e7b4123050fafb4d99)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:03.643316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268240c-e222-464d-91f0-789308bab3c9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:03.644958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 36/77] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/users/c987d6e5675d41e7b4123050fafb4d99 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:03.646953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602275-0295-4432-9b04-2f99d4125851 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.647629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602275-0295-4432-9b04-2f99d4125851 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.647940 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602275-0295-4432-9b04-2f99d4125851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.664283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-da602275-0295-4432-9b04-2f99d4125851 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:03.665459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-da602275-0295-4432-9b04-2f99d4125851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:03.672263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5312a8-f34f-43bf-bd16-38e7b7b5ab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.673031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5312a8-f34f-43bf-bd16-38e7b7b5ab08 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.673320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5312a8-f34f-43bf-bd16-38e7b7b5ab08 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.673600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5312a8-f34f-43bf-bd16-38e7b7b5ab08 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.674370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5312a8-f34f-43bf-bd16-38e7b7b5ab08 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:03.674687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5312a8-f34f-43bf-bd16-38e7b7b5ab08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:03.676073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5312a8-f34f-43bf-bd16-38e7b7b5ab08 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:03.683747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 42/78] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:03.684519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5312a8-f34f-43bf-bd16-38e7b7b5ab08 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.735837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 37/79] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:03.736626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da602275-0295-4432-9b04-2f99d4125851 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.739055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:03.789950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5312a8-f34f-43bf-bd16-38e7b7b5ab08 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.805746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:03.823269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e283c2-8530-4ca7-b69a-27d7f0fec7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.824070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e283c2-8530-4ca7-b69a-27d7f0fec7a1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.824358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e283c2-8530-4ca7-b69a-27d7f0fec7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.824640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e283c2-8530-4ca7-b69a-27d7f0fec7a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.834733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e283c2-8530-4ca7-b69a-27d7f0fec7a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:03.835447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da602275-0295-4432-9b04-2f99d4125851 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-18e283c2-8530-4ca7-b69a-27d7f0fec7a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:03.842882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:03.843890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:03.844674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 43/80] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:03.845290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e283c2-8530-4ca7-b69a-27d7f0fec7a1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:03.872877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd365c-e396-4b56-ab00-8aaa150dc63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.873530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd365c-e396-4b56-ab00-8aaa150dc63a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.873835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd365c-e396-4b56-ab00-8aaa150dc63a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.874128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd365c-e396-4b56-ab00-8aaa150dc63a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:03.938904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd365c-e396-4b56-ab00-8aaa150dc63a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:03.940377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd365c-e396-4b56-ab00-8aaa150dc63a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:03.943277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e283c2-8530-4ca7-b69a-27d7f0fec7a1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:03.953279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:03.976426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd216816-171a-4a13-85a1-9bcdbe58395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:03.977136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd216816-171a-4a13-85a1-9bcdbe58395f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:03.977426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd216816-171a-4a13-85a1-9bcdbe58395f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:03.977735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd216816-171a-4a13-85a1-9bcdbe58395f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.007617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:04.043727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 38/81] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:04.043727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd365c-e396-4b56-ab00-8aaa150dc63a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.045295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd216816-171a-4a13-85a1-9bcdbe58395f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.046805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd216816-171a-4a13-85a1-9bcdbe58395f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.139731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd365c-e396-4b56-ab00-8aaa150dc63a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:04.142159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 44/82] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:04.142996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd216816-171a-4a13-85a1-9bcdbe58395f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.153871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:04.173589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e507000-f887-45b7-b367-967632cdb979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:04.174498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e507000-f887-45b7-b367-967632cdb979 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:04.174786 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e507000-f887-45b7-b367-967632cdb979 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:04.175071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e507000-f887-45b7-b367-967632cdb979 None admin] PATH_INFO: `/v3/users/c987d6e5675d41e7b4123050fafb4d99` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.176272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e507000-f887-45b7-b367-967632cdb979 None admin] RBAC: Authorizing `identity:get_user(user_id=c987d6e5675d41e7b4123050fafb4d99)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.177743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e507000-f887-45b7-b367-967632cdb979 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.179433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 39/83] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/users/c987d6e5675d41e7b4123050fafb4d99 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:04.184617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e507000-f887-45b7-b367-967632cdb979 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.222388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:04.240526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd216816-171a-4a13-85a1-9bcdbe58395f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:04.274316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa612d6-b856-4b58-97aa-6329b0a522c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:04.275112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa612d6-b856-4b58-97aa-6329b0a522c3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:04.275407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa612d6-b856-4b58-97aa-6329b0a522c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:04.275698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa612d6-b856-4b58-97aa-6329b0a522c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.276863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa612d6-b856-4b58-97aa-6329b0a522c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.278354 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa612d6-b856-4b58-97aa-6329b0a522c3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.279992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 45/84] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:04.280833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa612d6-b856-4b58-97aa-6329b0a522c3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.283703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e507000-f887-45b7-b367-967632cdb979 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:04.318074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f62304-b012-48d9-a71e-b347d333d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:04.318750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62304-b012-48d9-a71e-b347d333d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:04.319044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62304-b012-48d9-a71e-b347d333d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:04.319327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62304-b012-48d9-a71e-b347d333d8f2 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.321202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f62304-b012-48d9-a71e-b347d333d8f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:04.321202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f62304-b012-48d9-a71e-b347d333d8f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.322189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f62304-b012-48d9-a71e-b347d333d8f2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.330123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 40/85] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:04.330775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f62304-b012-48d9-a71e-b347d333d8f2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.359680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:04.380260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa612d6-b856-4b58-97aa-6329b0a522c3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:04.413148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870a360-5105-4c92-ac1d-ed6618b1eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:04.414038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870a360-5105-4c92-ac1d-ed6618b1eda8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:04.414332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870a360-5105-4c92-ac1d-ed6618b1eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:04.414619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870a360-5105-4c92-ac1d-ed6618b1eda8 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.416335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870a360-5105-4c92-ac1d-ed6618b1eda8 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.418066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870a360-5105-4c92-ac1d-ed6618b1eda8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.419756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 46/86] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:04.424780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870a360-5105-4c92-ac1d-ed6618b1eda8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.427862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:04.430737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f62304-b012-48d9-a71e-b347d333d8f2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:04.465024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c60d-e8b1-46d7-8f5f-1227bcc39801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:04.465904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c60d-e8b1-46d7-8f5f-1227bcc39801 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:04.466225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c60d-e8b1-46d7-8f5f-1227bcc39801 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:04.466508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c60d-e8b1-46d7-8f5f-1227bcc39801 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.467712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4c60d-e8b1-46d7-8f5f-1227bcc39801 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.469165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4c60d-e8b1-46d7-8f5f-1227bcc39801 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.470915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 41/87] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:04.471660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c60d-e8b1-46d7-8f5f-1227bcc39801 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.526522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870a360-5105-4c92-ac1d-ed6618b1eda8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:04.560023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea593d2-59d6-4693-a28a-9c86a9419b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:04.560673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea593d2-59d6-4693-a28a-9c86a9419b00 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:04.560970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea593d2-59d6-4693-a28a-9c86a9419b00 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:04.561246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea593d2-59d6-4693-a28a-9c86a9419b00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.569463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c60d-e8b1-46d7-8f5f-1227bcc39801 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:04.575799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:04.602585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a82030-aeca-49f8-86c4-6be1130fae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:04.603866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a82030-aeca-49f8-86c4-6be1130fae66 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:04.604156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a82030-aeca-49f8-86c4-6be1130fae66 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:04.604493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a82030-aeca-49f8-86c4-6be1130fae66 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.605215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a82030-aeca-49f8-86c4-6be1130fae66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.606704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a82030-aeca-49f8-86c4-6be1130fae66 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.627049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea593d2-59d6-4693-a28a-9c86a9419b00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.628530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea593d2-59d6-4693-a28a-9c86a9419b00 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.706777 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 42/88] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:04 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:04.707712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a82030-aeca-49f8-86c4-6be1130fae66 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.732544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 47/89] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:04.733440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea593d2-59d6-4693-a28a-9c86a9419b00 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.741421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:04.785036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:04.828299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a82030-aeca-49f8-86c4-6be1130fae66 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:04.848319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea593d2-59d6-4693-a28a-9c86a9419b00 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:04.867957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3036ab6-7aab-4e50-b368-d24f839cad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:04.869208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3036ab6-7aab-4e50-b368-d24f839cad51 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:04.869495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3036ab6-7aab-4e50-b368-d24f839cad51 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:04.869777 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3036ab6-7aab-4e50-b368-d24f839cad51 None admin] PATH_INFO: `/v3/users/c987d6e5675d41e7b4123050fafb4d99` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.871081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3036ab6-7aab-4e50-b368-d24f839cad51 None admin] RBAC: Authorizing `identity:get_user(user_id=c987d6e5675d41e7b4123050fafb4d99)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.873105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3036ab6-7aab-4e50-b368-d24f839cad51 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.874873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 43/90] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/users/c987d6e5675d41e7b4123050fafb4d99 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:04.880285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3036ab6-7aab-4e50-b368-d24f839cad51 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:04.887668 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f946-1e57-4d77-ac90-c01ad20c5b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:04.888577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f946-1e57-4d77-ac90-c01ad20c5b69 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:04.889006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f946-1e57-4d77-ac90-c01ad20c5b69 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:04.889462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f946-1e57-4d77-ac90-c01ad20c5b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:04.890351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f946-1e57-4d77-ac90-c01ad20c5b69 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=ca796ca2de804171942d2631cccd56c7, user.id=e68f65e605c34489879af9763cddb11c) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:04.890665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f946-1e57-4d77-ac90-c01ad20c5b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:04.892157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f946-1e57-4d77-ac90-c01ad20c5b69 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:04.906305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 48/91] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/role_assignments?user.id=e68f65e605c34489879af9763cddb11c&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=fe4d636f5c4842f18569347911e6cdec => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:04.908529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f946-1e57-4d77-ac90-c01ad20c5b69 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:05.007088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:05.018903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3036ab6-7aab-4e50-b368-d24f839cad51 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:05.038207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f946-1e57-4d77-ac90-c01ad20c5b69 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:05.054545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:05.071564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053df4ff-62f5-4d0b-a05c-f56b89268fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:05.072803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053df4ff-62f5-4d0b-a05c-f56b89268fa3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:05.073085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053df4ff-62f5-4d0b-a05c-f56b89268fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:05.073368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053df4ff-62f5-4d0b-a05c-f56b89268fa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:05.074295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053df4ff-62f5-4d0b-a05c-f56b89268fa3 None admin] RBAC: Adding query filter params (role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=ca796ca2de804171942d2631cccd56c7, user.id=c987d6e5675d41e7b4123050fafb4d99) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:05.074611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053df4ff-62f5-4d0b-a05c-f56b89268fa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:05.076237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053df4ff-62f5-4d0b-a05c-f56b89268fa3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:05.082552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 44/92] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/role_assignments?user.id=c987d6e5675d41e7b4123050fafb4d99&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:05.083249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053df4ff-62f5-4d0b-a05c-f56b89268fa3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:05.086004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0588760-e0c8-4e38-9769-dd48378767d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:05.087452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0588760-e0c8-4e38-9769-dd48378767d6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:05.088143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0588760-e0c8-4e38-9769-dd48378767d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:05.088641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0588760-e0c8-4e38-9769-dd48378767d6 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:05.090695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0588760-e0c8-4e38-9769-dd48378767d6 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:05.092485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0588760-e0c8-4e38-9769-dd48378767d6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:05.094797 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 49/93] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:04 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:05.100991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0588760-e0c8-4e38-9769-dd48378767d6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:05.224413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053df4ff-62f5-4d0b-a05c-f56b89268fa3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:05.244579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0588760-e0c8-4e38-9769-dd48378767d6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:05.271695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b261735-a584-4adf-9236-24ead25420fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:05.273419 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b261735-a584-4adf-9236-24ead25420fb None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:05.273808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b261735-a584-4adf-9236-24ead25420fb None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:05.274200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b261735-a584-4adf-9236-24ead25420fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:05.292719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8be8-dadf-4d92-b6a8-748f51424fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:05.293668 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8be8-dadf-4d92-b6a8-748f51424fed None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:05.294165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8be8-dadf-4d92-b6a8-748f51424fed None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:05.294645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8be8-dadf-4d92-b6a8-748f51424fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:05.295693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f8be8-dadf-4d92-b6a8-748f51424fed None admin] RBAC: Adding query filter params (role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=ca796ca2de804171942d2631cccd56c7, user.id=c987d6e5675d41e7b4123050fafb4d99) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:05.296208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f8be8-dadf-4d92-b6a8-748f51424fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:05.298851 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f8be8-dadf-4d92-b6a8-748f51424fed None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:05.305505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 50/94] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:05 2022] GET /identity/v3/role_assignments?user.id=c987d6e5675d41e7b4123050fafb4d99&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=7583ad887d8b4052bd11783371c338b4 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:05.308386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:05.361727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b261735-a584-4adf-9236-24ead25420fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:05.363340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b261735-a584-4adf-9236-24ead25420fb None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:05.476493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 45/95] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:05.482421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8be8-dadf-4d92-b6a8-748f51424fed None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:05.594324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:05.670938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8be8-dadf-4d92-b6a8-748f51424fed None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:05.727341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a078413-a13e-4540-b134-4ab63c4bdab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:05.728782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a078413-a13e-4540-b134-4ab63c4bdab6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:05.729075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a078413-a13e-4540-b134-4ab63c4bdab6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:05.729357 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a078413-a13e-4540-b134-4ab63c4bdab6 None admin] PATH_INFO: `/v3/projects/5df1b615b14642ffb284dbfab9a1fee2` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:05.734644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a078413-a13e-4540-b134-4ab63c4bdab6 None admin] RBAC: Authorizing `identity:get_project(project_id=5df1b615b14642ffb284dbfab9a1fee2)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:05.736318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a078413-a13e-4540-b134-4ab63c4bdab6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:05.738280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 51/96] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:05 2022] GET /identity/v3/projects/5df1b615b14642ffb284dbfab9a1fee2 => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:05.744206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b261735-a584-4adf-9236-24ead25420fb None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:05.823152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:05.868111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b261735-a584-4adf-9236-24ead25420fb None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:05.904457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1d15b-ff88-4bba-a463-f61008bc2eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:05.905129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d15b-ff88-4bba-a463-f61008bc2eab None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:05.905425 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d15b-ff88-4bba-a463-f61008bc2eab None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:05.905713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d15b-ff88-4bba-a463-f61008bc2eab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:05.906601 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1d15b-ff88-4bba-a463-f61008bc2eab None admin] RBAC: Adding query filter params (role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=5df1b615b14642ffb284dbfab9a1fee2, user.id=c987d6e5675d41e7b4123050fafb4d99) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:05.907010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1d15b-ff88-4bba-a463-f61008bc2eab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:05.908482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1d15b-ff88-4bba-a463-f61008bc2eab None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:05.914745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 46/97] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:05 2022] GET /identity/v3/role_assignments?user.id=c987d6e5675d41e7b4123050fafb4d99&scope.project.id=5df1b615b14642ffb284dbfab9a1fee2&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:07.986906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76592547-af94-4f04-a9b8-c6bb3545a2ee None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:07.987895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76592547-af94-4f04-a9b8-c6bb3545a2ee None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:07.988385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76592547-af94-4f04-a9b8-c6bb3545a2ee None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:07.989983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 52/98] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:07.995349 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae527256-0b22-4f59-b094-7ab8e7f50b77 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:07.996132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae527256-0b22-4f59-b094-7ab8e7f50b77 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:07.996613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae527256-0b22-4f59-b094-7ab8e7f50b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.014242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ae527256-0b22-4f59-b094-7ab8e7f50b77 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:08.015749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ae527256-0b22-4f59-b094-7ab8e7f50b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:08.037122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f7a70-c4ee-4430-9a44-4e88185f7083 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:08.037696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f7a70-c4ee-4430-9a44-4e88185f7083 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:08.038195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f7a70-c4ee-4430-9a44-4e88185f7083 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.039746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 53/99] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:08.045537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5442ff5-7751-46ef-b57c-0e6ee3baff03 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:08.046105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5442ff5-7751-46ef-b57c-0e6ee3baff03 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:08.046582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5442ff5-7751-46ef-b57c-0e6ee3baff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.068715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b5442ff5-7751-46ef-b57c-0e6ee3baff03 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:08.070387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b5442ff5-7751-46ef-b57c-0e6ee3baff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:08.120933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 47/100] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:08.146663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae527256-0b22-4f59-b094-7ab8e7f50b77 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:08.213942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 54/101] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:08.238370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:08.257631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5442ff5-7751-46ef-b57c-0e6ee3baff03 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:08.322867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae527256-0b22-4f59-b094-7ab8e7f50b77 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:08.326509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:08.393798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411f78c-0627-4a34-9415-fd6fd1e9c125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:08.394638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411f78c-0627-4a34-9415-fd6fd1e9c125 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:08.394937 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411f78c-0627-4a34-9415-fd6fd1e9c125 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:08.395223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411f78c-0627-4a34-9415-fd6fd1e9c125 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.417080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5442ff5-7751-46ef-b57c-0e6ee3baff03 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:08.428486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411f78c-0627-4a34-9415-fd6fd1e9c125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-5411f78c-0627-4a34-9415-fd6fd1e9c125 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:08.437245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:08.438088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:08.438840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:08.444724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 48/102] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:08.444724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045cd7c-43eb-49a8-8b57-82a36de10ca9 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:08.444724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045cd7c-43eb-49a8-8b57-82a36de10ca9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:08.444724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045cd7c-43eb-49a8-8b57-82a36de10ca9 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.448782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 49/103] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:08.449644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045cd7c-43eb-49a8-8b57-82a36de10ca9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:08.487532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a53bc-386b-4542-8e07-0c8094eb2e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:08.488462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a53bc-386b-4542-8e07-0c8094eb2e31 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:08.488646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a53bc-386b-4542-8e07-0c8094eb2e31 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:08.488910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a53bc-386b-4542-8e07-0c8094eb2e31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.510750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a53bc-386b-4542-8e07-0c8094eb2e31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-bd5a53bc-386b-4542-8e07-0c8094eb2e31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:08.518706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:08.519550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:08.520283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 55/104] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:37:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:08.532789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb162b8-3415-42f6-8cab-bdee7ce5e5dc None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:08.533074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb162b8-3415-42f6-8cab-bdee7ce5e5dc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:08.533355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb162b8-3415-42f6-8cab-bdee7ce5e5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.627368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fcb162b8-3415-42f6-8cab-bdee7ce5e5dc None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:08.628567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fcb162b8-3415-42f6-8cab-bdee7ce5e5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:08.731519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045cd7c-43eb-49a8-8b57-82a36de10ca9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:08.743267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:08.772103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7197-1a6c-40ff-9d8b-462579602511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:08.775249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7197-1a6c-40ff-9d8b-462579602511 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:08.775551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7197-1a6c-40ff-9d8b-462579602511 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:08.775845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7197-1a6c-40ff-9d8b-462579602511 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.776613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7197-1a6c-40ff-9d8b-462579602511 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:08.776932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7197-1a6c-40ff-9d8b-462579602511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:08.778399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7197-1a6c-40ff-9d8b-462579602511 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:08.779842 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 56/105] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:08.780575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb162b8-3415-42f6-8cab-bdee7ce5e5dc None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:08.798789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 50/106] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:08.807450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7197-1a6c-40ff-9d8b-462579602511 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:08.841214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:08.944967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb162b8-3415-42f6-8cab-bdee7ce5e5dc None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:08.950310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7197-1a6c-40ff-9d8b-462579602511 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:08.997945 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf59a1-54c3-4a9e-804d-d0aeccd82cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:08.998462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad5fe3-9795-45e4-8118-f828afb3167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:08.998816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf59a1-54c3-4a9e-804d-d0aeccd82cd3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:08.998816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf59a1-54c3-4a9e-804d-d0aeccd82cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:08.999150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5fe3-9795-45e4-8118-f828afb3167f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:08.999245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf59a1-54c3-4a9e-804d-d0aeccd82cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.999390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5fe3-9795-45e4-8118-f828afb3167f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:08.999650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5fe3-9795-45e4-8118-f828afb3167f None admin] PATH_INFO: `/v3/users/0d2f3620f7b6460ab31d4270adba0f11` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:08.999860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf59a1-54c3-4a9e-804d-d0aeccd82cd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:09.000220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf59a1-54c3-4a9e-804d-d0aeccd82cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:09.001591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf59a1-54c3-4a9e-804d-d0aeccd82cd3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:09.011223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 57/107] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:09.012052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf59a1-54c3-4a9e-804d-d0aeccd82cd3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.019234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad5fe3-9795-45e4-8118-f828afb3167f None admin] RBAC: Authorizing `identity:get_user(user_id=0d2f3620f7b6460ab31d4270adba0f11)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:09.020733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad5fe3-9795-45e4-8118-f828afb3167f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:09.022490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 51/108] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:08 2022] GET /identity/v3/users/0d2f3620f7b6460ab31d4270adba0f11 => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:09.023734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc18a77-9100-4782-bcf3-76e82befdde8 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.024027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc18a77-9100-4782-bcf3-76e82befdde8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.024308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc18a77-9100-4782-bcf3-76e82befdde8 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.026518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 52/109] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:09.027540 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf20fe3-52df-4a3a-bee6-a053bf331aa4 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.027873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf20fe3-52df-4a3a-bee6-a053bf331aa4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.028163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf20fe3-52df-4a3a-bee6-a053bf331aa4 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.029401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 53/110] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:09.030838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8fa7-79d2-4e86-a988-553a6405f1e6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.030954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8fa7-79d2-4e86-a988-553a6405f1e6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.031151 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8fa7-79d2-4e86-a988-553a6405f1e6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.032374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 54/111] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:09.033074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d8fa7-79d2-4e86-a988-553a6405f1e6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.063587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:09.106697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:09.114954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf59a1-54c3-4a9e-804d-d0aeccd82cd3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:09.135100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d8fa7-79d2-4e86-a988-553a6405f1e6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:09.147937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643c870-fbbb-4129-be09-72b155dd18b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:09.148737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643c870-fbbb-4129-be09-72b155dd18b9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.149027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643c870-fbbb-4129-be09-72b155dd18b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.149308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643c870-fbbb-4129-be09-72b155dd18b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.151038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643c870-fbbb-4129-be09-72b155dd18b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:09.152492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643c870-fbbb-4129-be09-72b155dd18b9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:09.154125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 58/112] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:09.154891 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643c870-fbbb-4129-be09-72b155dd18b9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.173734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d4af0-9a8e-45f4-8f7e-0973078754ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:09.174648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d4af0-9a8e-45f4-8f7e-0973078754ba None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.174948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d4af0-9a8e-45f4-8f7e-0973078754ba None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.175222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d4af0-9a8e-45f4-8f7e-0973078754ba None admin] PATH_INFO: `/v3/users/e68f65e605c34489879af9763cddb11c` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.176414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d4af0-9a8e-45f4-8f7e-0973078754ba None admin] RBAC: Authorizing `identity:get_user(user_id=e68f65e605c34489879af9763cddb11c)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:09.177867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d4af0-9a8e-45f4-8f7e-0973078754ba None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:09.179643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 55/113] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:09 2022] GET /identity/v3/users/e68f65e605c34489879af9763cddb11c => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:09.180696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3065-26c7-403c-9bad-863b8ee0ebfb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.181546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3065-26c7-403c-9bad-863b8ee0ebfb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.181841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3065-26c7-403c-9bad-863b8ee0ebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.197714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-917b3065-26c7-403c-9bad-863b8ee0ebfb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:09.198943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-917b3065-26c7-403c-9bad-863b8ee0ebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:09.256060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643c870-fbbb-4129-be09-72b155dd18b9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:09.264345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 56/114] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:09.265806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0088603-4b11-4b01-93e2-e6f33a684abd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.266138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0088603-4b11-4b01-93e2-e6f33a684abd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.266430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0088603-4b11-4b01-93e2-e6f33a684abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.288734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7ac1a-9502-4330-976e-4fbacdbd14c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:09.289390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7ac1a-9502-4330-976e-4fbacdbd14c7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.289680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7ac1a-9502-4330-976e-4fbacdbd14c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.289994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7ac1a-9502-4330-976e-4fbacdbd14c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.293224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d0088603-4b11-4b01-93e2-e6f33a684abd None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:09.294473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d0088603-4b11-4b01-93e2-e6f33a684abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:09.317603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:09.318148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:09.355982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7ac1a-9502-4330-976e-4fbacdbd14c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:09.357458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7ac1a-9502-4330-976e-4fbacdbd14c7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:09.359983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 57/115] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:09.361035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ade7f-7f74-4241-a48e-ad5f4af855cf None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.361332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ade7f-7f74-4241-a48e-ad5f4af855cf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.361619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ade7f-7f74-4241-a48e-ad5f4af855cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.378001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6e8ade7f-7f74-4241-a48e-ad5f4af855cf None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:09.380002 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6e8ade7f-7f74-4241-a48e-ad5f4af855cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:09.448489 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 58/116] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:09.449384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ade7f-7f74-4241-a48e-ad5f4af855cf None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.459309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 59/117] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:09.460198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7ac1a-9502-4330-976e-4fbacdbd14c7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.522573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:09.539742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:09.551823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ade7f-7f74-4241-a48e-ad5f4af855cf None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:09.564440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7ac1a-9502-4330-976e-4fbacdbd14c7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:09.585483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad07e3-ba8c-474b-a80b-88246ed3ad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:09.586175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad07e3-ba8c-474b-a80b-88246ed3ad08 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.586467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad07e3-ba8c-474b-a80b-88246ed3ad08 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.586745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad07e3-ba8c-474b-a80b-88246ed3ad08 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.587450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad07e3-ba8c-474b-a80b-88246ed3ad08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:09.588900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad07e3-ba8c-474b-a80b-88246ed3ad08 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:09.597050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bbc35-ebe2-441e-8301-720435d6aef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:09.597933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbc35-ebe2-441e-8301-720435d6aef8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.598225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbc35-ebe2-441e-8301-720435d6aef8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.598508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbc35-ebe2-441e-8301-720435d6aef8 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.599689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bbc35-ebe2-441e-8301-720435d6aef8 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:09.601135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bbc35-ebe2-441e-8301-720435d6aef8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:09.603386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 60/118] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:09 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:09.604147 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bbc35-ebe2-441e-8301-720435d6aef8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.679243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 59/119] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:37:09 2022] POST /identity/v3/projects => generated 302 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:09.680073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad07e3-ba8c-474b-a80b-88246ed3ad08 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.703165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bbc35-ebe2-441e-8301-720435d6aef8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:09.736390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:09.745545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3daa8c-049e-4807-862f-36eb1f870027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:09.751764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3daa8c-049e-4807-862f-36eb1f870027 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.751764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3daa8c-049e-4807-862f-36eb1f870027 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.751764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3daa8c-049e-4807-862f-36eb1f870027 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.760742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3daa8c-049e-4807-862f-36eb1f870027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-cb3daa8c-049e-4807-862f-36eb1f870027 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:09.769206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:09.770157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:09.770914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 61/120] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:09.777077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3daa8c-049e-4807-862f-36eb1f870027 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.809424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad07e3-ba8c-474b-a80b-88246ed3ad08 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:09.851584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21a5b6-bcfd-4abe-ba77-6588e99c8fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:09.853671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21a5b6-bcfd-4abe-ba77-6588e99c8fe9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.854342 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21a5b6-bcfd-4abe-ba77-6588e99c8fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.854631 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21a5b6-bcfd-4abe-ba77-6588e99c8fe9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.865313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:09.872114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21a5b6-bcfd-4abe-ba77-6588e99c8fe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-bc21a5b6-bcfd-4abe-ba77-6588e99c8fe9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:09.881317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:09.882842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 60/121] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:09.883598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21a5b6-bcfd-4abe-ba77-6588e99c8fe9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.893476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3daa8c-049e-4807-862f-36eb1f870027 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:09.932551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba4ec8-b3b9-4b91-a775-1cf5296f50cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:09.933520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4ec8-b3b9-4b91-a775-1cf5296f50cb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:09.933809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4ec8-b3b9-4b91-a775-1cf5296f50cb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:09.934118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4ec8-b3b9-4b91-a775-1cf5296f50cb None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:09.935410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba4ec8-b3b9-4b91-a775-1cf5296f50cb None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:09.936863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba4ec8-b3b9-4b91-a775-1cf5296f50cb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:09.938579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 62/122] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:09 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:09.939314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba4ec8-b3b9-4b91-a775-1cf5296f50cb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:09.995681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:10.003862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21a5b6-bcfd-4abe-ba77-6588e99c8fe9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.043485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a96ba-f176-4240-8e36-aa2133c236e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.044091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba4ec8-b3b9-4b91-a775-1cf5296f50cb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.044515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a96ba-f176-4240-8e36-aa2133c236e4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.044588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a96ba-f176-4240-8e36-aa2133c236e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.044820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a96ba-f176-4240-8e36-aa2133c236e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.056546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a96ba-f176-4240-8e36-aa2133c236e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6c3a96ba-f176-4240-8e36-aa2133c236e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:10.064363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:10.065187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 61/123] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:10.066735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a96ba-f176-4240-8e36-aa2133c236e4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:10.082374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dfa68-929c-40db-977b-84f12f8fda76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.083531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dfa68-929c-40db-977b-84f12f8fda76 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.083883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dfa68-929c-40db-977b-84f12f8fda76 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.084164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74dfa68-929c-40db-977b-84f12f8fda76 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7/users/e68f65e605c34489879af9763cddb11c/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.086196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dfa68-929c-40db-977b-84f12f8fda76 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca796ca2de804171942d2631cccd56c7, user_id=e68f65e605c34489879af9763cddb11c, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:10.088335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74dfa68-929c-40db-977b-84f12f8fda76 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:10.130315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:10.164397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 63/124] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:37:09 2022] PUT /identity/v3/projects/ca796ca2de804171942d2631cccd56c7/users/e68f65e605c34489879af9763cddb11c/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:10.172296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a96ba-f176-4240-8e36-aa2133c236e4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.173970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dfa68-929c-40db-977b-84f12f8fda76 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:10.214907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be3f0-d952-4239-b068-bb3a5f31f39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.215776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be3f0-d952-4239-b068-bb3a5f31f39e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.216081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be3f0-d952-4239-b068-bb3a5f31f39e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.216432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be3f0-d952-4239-b068-bb3a5f31f39e None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.217274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be3f0-d952-4239-b068-bb3a5f31f39e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:10.217655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be3f0-d952-4239-b068-bb3a5f31f39e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:10.219254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be3f0-d952-4239-b068-bb3a5f31f39e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:10.232144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 62/125] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:10.234184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be3f0-d952-4239-b068-bb3a5f31f39e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:10.317316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74dfa68-929c-40db-977b-84f12f8fda76 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.340469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:10.370659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5c742-26f5-4f3c-b900-e35e6b70b788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.371364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5c742-26f5-4f3c-b900-e35e6b70b788 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.371659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5c742-26f5-4f3c-b900-e35e6b70b788 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.371950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5c742-26f5-4f3c-b900-e35e6b70b788 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.372724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5c742-26f5-4f3c-b900-e35e6b70b788 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:10.373203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5c742-26f5-4f3c-b900-e35e6b70b788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:10.374657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5c742-26f5-4f3c-b900-e35e6b70b788 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:10.382545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be3f0-d952-4239-b068-bb3a5f31f39e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.386710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 64/126] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:10.387769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5c742-26f5-4f3c-b900-e35e6b70b788 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:10.409168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:10.428156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdb27d-6b90-4f73-ad08-f22ce993a6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.429290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdb27d-6b90-4f73-ad08-f22ce993a6c4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.429585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdb27d-6b90-4f73-ad08-f22ce993a6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.430163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdb27d-6b90-4f73-ad08-f22ce993a6c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.431865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdb27d-6b90-4f73-ad08-f22ce993a6c4 None admin] RBAC: Adding query filter params (role.id=3394eca9fdfe4986becd456e050c26e3, scope.project.id=ca796ca2de804171942d2631cccd56c7, user.id=0d2f3620f7b6460ab31d4270adba0f11) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:10.432196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdb27d-6b90-4f73-ad08-f22ce993a6c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:10.434408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdb27d-6b90-4f73-ad08-f22ce993a6c4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:10.441567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 63/127] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/role_assignments?user.id=0d2f3620f7b6460ab31d4270adba0f11&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=3394eca9fdfe4986becd456e050c26e3 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:10.443252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdb27d-6b90-4f73-ad08-f22ce993a6c4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:10.525676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5c742-26f5-4f3c-b900-e35e6b70b788 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.566164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdb27d-6b90-4f73-ad08-f22ce993a6c4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.574862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21b668-f0cc-4370-ba29-b3aeb091ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.575641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21b668-f0cc-4370-ba29-b3aeb091ee21 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.576096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21b668-f0cc-4370-ba29-b3aeb091ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.576485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21b668-f0cc-4370-ba29-b3aeb091ee21 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.577404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21b668-f0cc-4370-ba29-b3aeb091ee21 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:10.577818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21b668-f0cc-4370-ba29-b3aeb091ee21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:10.579371 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21b668-f0cc-4370-ba29-b3aeb091ee21 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:10.589360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 65/128] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:10.594692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21b668-f0cc-4370-ba29-b3aeb091ee21 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:10.609568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78d72b-3853-4f4b-bebe-dbcee32e71c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.610819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78d72b-3853-4f4b-bebe-dbcee32e71c5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.611113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78d72b-3853-4f4b-bebe-dbcee32e71c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.611398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78d72b-3853-4f4b-bebe-dbcee32e71c5 None admin] PATH_INFO: `/v3/users/c987d6e5675d41e7b4123050fafb4d99` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.613569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78d72b-3853-4f4b-bebe-dbcee32e71c5 None admin] RBAC: Authorizing `identity:get_user(user_id=c987d6e5675d41e7b4123050fafb4d99)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:10.615399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78d72b-3853-4f4b-bebe-dbcee32e71c5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:10.617866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 64/129] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/users/c987d6e5675d41e7b4123050fafb4d99 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:10.619315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78d72b-3853-4f4b-bebe-dbcee32e71c5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:10.644722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:10.696739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:10.753068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78d72b-3853-4f4b-bebe-dbcee32e71c5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.754191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21b668-f0cc-4370-ba29-b3aeb091ee21 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.797816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168b44-66a6-4bce-a43a-b2dc8ff0ce9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.799148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168b44-66a6-4bce-a43a-b2dc8ff0ce9b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.799881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168b44-66a6-4bce-a43a-b2dc8ff0ce9b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.800383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168b44-66a6-4bce-a43a-b2dc8ff0ce9b None admin] PATH_INFO: `/v3/users/c987d6e5675d41e7b4123050fafb4d99` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.801753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f168b44-66a6-4bce-a43a-b2dc8ff0ce9b None admin] RBAC: Authorizing `identity:get_user(user_id=c987d6e5675d41e7b4123050fafb4d99)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:10.803375 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f168b44-66a6-4bce-a43a-b2dc8ff0ce9b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:10.804986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 65/130] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/users/c987d6e5675d41e7b4123050fafb4d99 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:10.805540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5944e-603b-4411-b86f-4dd6737a4e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.805767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168b44-66a6-4bce-a43a-b2dc8ff0ce9b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:10.806505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5944e-603b-4411-b86f-4dd6737a4e18 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.806792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5944e-603b-4411-b86f-4dd6737a4e18 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.807135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5944e-603b-4411-b86f-4dd6737a4e18 None admin] PATH_INFO: `/v3/users/c987d6e5675d41e7b4123050fafb4d99` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.808400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5944e-603b-4411-b86f-4dd6737a4e18 None admin] RBAC: Authorizing `identity:get_user(user_id=c987d6e5675d41e7b4123050fafb4d99)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:10.809945 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5944e-603b-4411-b86f-4dd6737a4e18 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:10.811574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 66/131] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/users/c987d6e5675d41e7b4123050fafb4d99 => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:10.812326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5944e-603b-4411-b86f-4dd6737a4e18 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:10.925794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168b44-66a6-4bce-a43a-b2dc8ff0ce9b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.956364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5944e-603b-4411-b86f-4dd6737a4e18 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:10.957020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:10.971591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced782cd-61bc-4bce-b434-97f251708b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:10.972577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced782cd-61bc-4bce-b434-97f251708b7d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:10.980724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced782cd-61bc-4bce-b434-97f251708b7d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:10.980724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced782cd-61bc-4bce-b434-97f251708b7d None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:10.980905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:10.981000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced782cd-61bc-4bce-b434-97f251708b7d None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:10.981000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced782cd-61bc-4bce-b434-97f251708b7d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:10.981511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 66/132] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:10.982353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced782cd-61bc-4bce-b434-97f251708b7d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:11.019960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f798d-070b-49ff-8aed-5a06f4fcaf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:11.020878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f798d-070b-49ff-8aed-5a06f4fcaf55 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:11.021167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f798d-070b-49ff-8aed-5a06f4fcaf55 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:11.021451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f798d-070b-49ff-8aed-5a06f4fcaf55 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:11.023144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f798d-070b-49ff-8aed-5a06f4fcaf55 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:11.024665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f798d-070b-49ff-8aed-5a06f4fcaf55 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:11.027217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 67/133] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:11.027841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f798d-070b-49ff-8aed-5a06f4fcaf55 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:11.133455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced782cd-61bc-4bce-b434-97f251708b7d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:11.169312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d3f39-84d1-443b-9c1c-72d370095ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:11.170347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d3f39-84d1-443b-9c1c-72d370095ffd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:11.170642 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d3f39-84d1-443b-9c1c-72d370095ffd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:11.170936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d3f39-84d1-443b-9c1c-72d370095ffd None admin] PATH_INFO: `/v3/projects/5df1b615b14642ffb284dbfab9a1fee2` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:11.172204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d3f39-84d1-443b-9c1c-72d370095ffd None admin] RBAC: Authorizing `identity:get_project(project_id=5df1b615b14642ffb284dbfab9a1fee2)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:11.173878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d3f39-84d1-443b-9c1c-72d370095ffd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:11.176038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 67/134] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:10 2022] GET /identity/v3/projects/5df1b615b14642ffb284dbfab9a1fee2 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:11.176741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d3f39-84d1-443b-9c1c-72d370095ffd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:11.244935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f798d-070b-49ff-8aed-5a06f4fcaf55 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:11.254304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:11.290807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d3f39-84d1-443b-9c1c-72d370095ffd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:11.310948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ca45a-e3f6-453c-b452-fc837964b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:11.311951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ca45a-e3f6-453c-b452-fc837964b742 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:11.312353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ca45a-e3f6-453c-b452-fc837964b742 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:11.312676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ca45a-e3f6-453c-b452-fc837964b742 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7/users/c987d6e5675d41e7b4123050fafb4d99/roles/7583ad887d8b4052bd11783371c338b4` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:11.315017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ca45a-e3f6-453c-b452-fc837964b742 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca796ca2de804171942d2631cccd56c7, user_id=c987d6e5675d41e7b4123050fafb4d99, role_id=7583ad887d8b4052bd11783371c338b4)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:11.316560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ca45a-e3f6-453c-b452-fc837964b742 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:11.326932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a4eb8-f677-4c02-bbb6-a880e8e92118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:11.328544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a4eb8-f677-4c02-bbb6-a880e8e92118 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:11.328920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a4eb8-f677-4c02-bbb6-a880e8e92118 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:11.329307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a4eb8-f677-4c02-bbb6-a880e8e92118 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7/users/c987d6e5675d41e7b4123050fafb4d99/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:11.331636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a4eb8-f677-4c02-bbb6-a880e8e92118 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca796ca2de804171942d2631cccd56c7, user_id=c987d6e5675d41e7b4123050fafb4d99, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:11.333557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a4eb8-f677-4c02-bbb6-a880e8e92118 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:11.417611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 68/135] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:37:11 2022] PUT /identity/v3/projects/ca796ca2de804171942d2631cccd56c7/users/c987d6e5675d41e7b4123050fafb4d99/roles/7583ad887d8b4052bd11783371c338b4 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:11.418542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ca45a-e3f6-453c-b452-fc837964b742 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:11.452832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 68/136] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:37:11 2022] PUT /identity/v3/projects/ca796ca2de804171942d2631cccd56c7/users/c987d6e5675d41e7b4123050fafb4d99/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:11.459563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:11.534084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ca45a-e3f6-453c-b452-fc837964b742 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:11.576990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9b6e-ceb0-4a97-8cd7-1b9881a68704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:11.577960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9b6e-ceb0-4a97-8cd7-1b9881a68704 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:11.578250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9b6e-ceb0-4a97-8cd7-1b9881a68704 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:11.578537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9b6e-ceb0-4a97-8cd7-1b9881a68704 None admin] PATH_INFO: `/v3/projects/5df1b615b14642ffb284dbfab9a1fee2/users/c987d6e5675d41e7b4123050fafb4d99/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:11.580631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9b6e-ceb0-4a97-8cd7-1b9881a68704 None admin] RBAC: Authorizing `identity:create_grant(project_id=5df1b615b14642ffb284dbfab9a1fee2, user_id=c987d6e5675d41e7b4123050fafb4d99, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:11.586364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9b6e-ceb0-4a97-8cd7-1b9881a68704 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:11.668913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 69/137] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:37:11 2022] PUT /identity/v3/projects/5df1b615b14642ffb284dbfab9a1fee2/users/c987d6e5675d41e7b4123050fafb4d99/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:12.308649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bae34-2313-405d-883c-e22ddb9bff53 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:12.309646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bae34-2313-405d-883c-e22ddb9bff53 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:12.310199 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bae34-2313-405d-883c-e22ddb9bff53 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:12.311826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 69/138] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:12.323657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb0cb5-df9a-4eb2-afc1-c306ff12de60 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:12.324584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb0cb5-df9a-4eb2-afc1-c306ff12de60 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:12.325092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb0cb5-df9a-4eb2-afc1-c306ff12de60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:12.350883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-57eb0cb5-df9a-4eb2-afc1-c306ff12de60 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:12.352421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-57eb0cb5-df9a-4eb2-afc1-c306ff12de60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:12.441992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 70/139] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:12.488587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bae34-2313-405d-883c-e22ddb9bff53 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:12.623614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bae34-2313-405d-883c-e22ddb9bff53 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:12.646766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:12.670452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216df98b-bf5f-4d66-8b8a-d4aecc36556d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:12.671634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216df98b-bf5f-4d66-8b8a-d4aecc36556d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:12.672161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216df98b-bf5f-4d66-8b8a-d4aecc36556d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:12.672662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216df98b-bf5f-4d66-8b8a-d4aecc36556d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:12.674138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216df98b-bf5f-4d66-8b8a-d4aecc36556d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:12.675902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216df98b-bf5f-4d66-8b8a-d4aecc36556d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:12.677822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 70/140] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:12.683588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb0cb5-df9a-4eb2-afc1-c306ff12de60 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:12.716375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:12.822060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb0cb5-df9a-4eb2-afc1-c306ff12de60 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:12.865673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b51aa4-8a95-4e82-9c96-a8436f926198 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:12.866447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b51aa4-8a95-4e82-9c96-a8436f926198 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:12.866945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b51aa4-8a95-4e82-9c96-a8436f926198 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:12.869148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 71/141] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:12.874685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ba22-2bed-4a14-bdb6-a682c0511516 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:12.874926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ba22-2bed-4a14-bdb6-a682c0511516 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:12.875235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74ba22-2bed-4a14-bdb6-a682c0511516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:12.879516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd344-4bc4-421c-b1a3-6eb8c2c393a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:12.880509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd344-4bc4-421c-b1a3-6eb8c2c393a0 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:12.880990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd344-4bc4-421c-b1a3-6eb8c2c393a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:12.881569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd344-4bc4-421c-b1a3-6eb8c2c393a0 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:12.882495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcd344-4bc4-421c-b1a3-6eb8c2c393a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:12.884853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcd344-4bc4-421c-b1a3-6eb8c2c393a0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:12.917358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7e74ba22-2bed-4a14-bdb6-a682c0511516 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:12.918605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7e74ba22-2bed-4a14-bdb6-a682c0511516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:13.013012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 71/142] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:12 2022] POST /identity/v3/users => generated 283 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:13.034321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 72/143] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:13.076085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd344-4bc4-421c-b1a3-6eb8c2c393a0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:13.183219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:13.202793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd344-4bc4-421c-b1a3-6eb8c2c393a0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:13.252253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4a4b9-2e28-4c05-8c18-f4a4bbfde197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:13.253089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4a4b9-2e28-4c05-8c18-f4a4bbfde197 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:13.253372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4a4b9-2e28-4c05-8c18-f4a4bbfde197 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:13.253659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4a4b9-2e28-4c05-8c18-f4a4bbfde197 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:13.270848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4a4b9-2e28-4c05-8c18-f4a4bbfde197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d5d4a4b9-2e28-4c05-8c18-f4a4bbfde197 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:13.278831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:13.279733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:13.280487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 72/144] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:37:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:13.287725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ba22-2bed-4a14-bdb6-a682c0511516 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:13.454840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74ba22-2bed-4a14-bdb6-a682c0511516 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:13.471920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:13.496854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b76e43-f288-43c9-9b47-5d0ab5878a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:13.497533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b76e43-f288-43c9-9b47-5d0ab5878a8c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:13.497813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b76e43-f288-43c9-9b47-5d0ab5878a8c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:13.498112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b76e43-f288-43c9-9b47-5d0ab5878a8c None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:13.498860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b76e43-f288-43c9-9b47-5d0ab5878a8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:13.499163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b76e43-f288-43c9-9b47-5d0ab5878a8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:13.500541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b76e43-f288-43c9-9b47-5d0ab5878a8c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:13.515823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 73/145] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:13.521366 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4a4b9-2e28-4c05-8c18-f4a4bbfde197 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:13.655955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4a4b9-2e28-4c05-8c18-f4a4bbfde197 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:13.700701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91873e5d-14a8-45f9-9faf-befb025d3cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:13.701581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91873e5d-14a8-45f9-9faf-befb025d3cfc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:13.701877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91873e5d-14a8-45f9-9faf-befb025d3cfc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:13.702171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91873e5d-14a8-45f9-9faf-befb025d3cfc None admin] PATH_INFO: `/v3/users/e68f65e605c34489879af9763cddb11c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:13.703371 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91873e5d-14a8-45f9-9faf-befb025d3cfc None admin] RBAC: Authorizing `identity:get_user(user_id=e68f65e605c34489879af9763cddb11c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:13.704830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91873e5d-14a8-45f9-9faf-befb025d3cfc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:13.706706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 73/146] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:13 2022] GET /identity/v3/users/e68f65e605c34489879af9763cddb11c => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:13.715016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b76e43-f288-43c9-9b47-5d0ab5878a8c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:13.798517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c87184-16df-46ad-915d-d989d90708e1 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:13.798840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c87184-16df-46ad-915d-d989d90708e1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:13.799092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c87184-16df-46ad-915d-d989d90708e1 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:13.800614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 74/147] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:13.809500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaa154-a062-4f24-8c3d-27cfb15b8c78 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:13.809793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaa154-a062-4f24-8c3d-27cfb15b8c78 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:13.810099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eaa154-a062-4f24-8c3d-27cfb15b8c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:13.846163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-31eaa154-a062-4f24-8c3d-27cfb15b8c78 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:13.847353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-31eaa154-a062-4f24-8c3d-27cfb15b8c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:13.857716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:13.904658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b76e43-f288-43c9-9b47-5d0ab5878a8c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:13.936572 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 75/148] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 03 09:37:13.954966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af66c4-d818-42dc-9e2a-f8bb7506bdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:13.955618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af66c4-d818-42dc-9e2a-f8bb7506bdd5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:13.955903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af66c4-d818-42dc-9e2a-f8bb7506bdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:13.956182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af66c4-d818-42dc-9e2a-f8bb7506bdd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:13.974594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eaa154-a062-4f24-8c3d-27cfb15b8c78 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:14.038784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:14.078230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af66c4-d818-42dc-9e2a-f8bb7506bdd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:14.079710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af66c4-d818-42dc-9e2a-f8bb7506bdd5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:14.156375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eaa154-a062-4f24-8c3d-27cfb15b8c78 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:14.224555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9459-04a2-4908-b87b-272f4bb64e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:14.225420 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9459-04a2-4908-b87b-272f4bb64e8d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:14.225641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9459-04a2-4908-b87b-272f4bb64e8d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:14.225929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c9459-04a2-4908-b87b-272f4bb64e8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:14.234986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:14.238495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c9459-04a2-4908-b87b-272f4bb64e8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8f8c9459-04a2-4908-b87b-272f4bb64e8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:14.249644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:14.251417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:14.252166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 76/149] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:14.260075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9459-04a2-4908-b87b-272f4bb64e8d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:14.282166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 74/150] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:14.288165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af66c4-d818-42dc-9e2a-f8bb7506bdd5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:14.382760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c9459-04a2-4908-b87b-272f4bb64e8d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:14.412596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:14.433467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76d0d0-2124-4c7c-b071-751b4822cf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:14.434214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76d0d0-2124-4c7c-b071-751b4822cf60 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:14.434499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76d0d0-2124-4c7c-b071-751b4822cf60 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:14.434774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76d0d0-2124-4c7c-b071-751b4822cf60 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:14.435525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76d0d0-2124-4c7c-b071-751b4822cf60 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:14.435923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76d0d0-2124-4c7c-b071-751b4822cf60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:14.438097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76d0d0-2124-4c7c-b071-751b4822cf60 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:14.453212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 77/151] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:14.459186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76d0d0-2124-4c7c-b071-751b4822cf60 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:14.461430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af66c4-d818-42dc-9e2a-f8bb7506bdd5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:14.527451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69124b0d-3895-4631-b599-bf5719109189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:14.528255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69124b0d-3895-4631-b599-bf5719109189 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:14.528533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69124b0d-3895-4631-b599-bf5719109189 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:14.528807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69124b0d-3895-4631-b599-bf5719109189 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:14.530991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69124b0d-3895-4631-b599-bf5719109189 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:14.537355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69124b0d-3895-4631-b599-bf5719109189 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:14.539867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 75/152] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:14 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:14.545980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69124b0d-3895-4631-b599-bf5719109189 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:14.573165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:14.621723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76d0d0-2124-4c7c-b071-751b4822cf60 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:14.670567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93c792-5d0d-40af-b221-afdea7c2db62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:14.671538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93c792-5d0d-40af-b221-afdea7c2db62 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:14.671829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93c792-5d0d-40af-b221-afdea7c2db62 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:14.672106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93c792-5d0d-40af-b221-afdea7c2db62 None admin] PATH_INFO: `/v3/users/0d2f3620f7b6460ab31d4270adba0f11` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:14.674691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93c792-5d0d-40af-b221-afdea7c2db62 None admin] RBAC: Authorizing `identity:get_user(user_id=0d2f3620f7b6460ab31d4270adba0f11)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:14.676310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93c792-5d0d-40af-b221-afdea7c2db62 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:14.678398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 78/153] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:14 2022] GET /identity/v3/users/0d2f3620f7b6460ab31d4270adba0f11 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:14.683563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93c792-5d0d-40af-b221-afdea7c2db62 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:14.690742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69124b0d-3895-4631-b599-bf5719109189 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:14.735897 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b16fe-1710-47b4-9180-b3c5752cf922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:14.736823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b16fe-1710-47b4-9180-b3c5752cf922 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:14.737353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b16fe-1710-47b4-9180-b3c5752cf922 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:14.737787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b16fe-1710-47b4-9180-b3c5752cf922 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:14.738796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b16fe-1710-47b4-9180-b3c5752cf922 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=ca796ca2de804171942d2631cccd56c7, user.id=e68f65e605c34489879af9763cddb11c) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:14.739263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b16fe-1710-47b4-9180-b3c5752cf922 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:14.740861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b16fe-1710-47b4-9180-b3c5752cf922 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:14.748229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 76/154] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:14 2022] GET /identity/v3/role_assignments?user.id=e68f65e605c34489879af9763cddb11c&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=fe4d636f5c4842f18569347911e6cdec => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:14.824720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93c792-5d0d-40af-b221-afdea7c2db62 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:14.857036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:14.869507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d3499-1249-401b-ac7f-387edc69d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:14.870437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d3499-1249-401b-ac7f-387edc69d62d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:14.870721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d3499-1249-401b-ac7f-387edc69d62d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:14.870995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d3499-1249-401b-ac7f-387edc69d62d None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:14.872308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d3499-1249-401b-ac7f-387edc69d62d None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:14.873795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d3499-1249-401b-ac7f-387edc69d62d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:14.875814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 79/155] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:14 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:14.881451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b16fe-1710-47b4-9180-b3c5752cf922 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:14.904438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:15.033194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b16fe-1710-47b4-9180-b3c5752cf922 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:15.078866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de07f0f-e581-47bb-8ec8-15d7e3fd87c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:15.079698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de07f0f-e581-47bb-8ec8-15d7e3fd87c1 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:15.079981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de07f0f-e581-47bb-8ec8-15d7e3fd87c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:15.080263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de07f0f-e581-47bb-8ec8-15d7e3fd87c1 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7/users/0d2f3620f7b6460ab31d4270adba0f11/roles/3394eca9fdfe4986becd456e050c26e3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:15.082290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de07f0f-e581-47bb-8ec8-15d7e3fd87c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca796ca2de804171942d2631cccd56c7, user_id=0d2f3620f7b6460ab31d4270adba0f11, role_id=3394eca9fdfe4986becd456e050c26e3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:15.083744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de07f0f-e581-47bb-8ec8-15d7e3fd87c1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:15.092658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c9aa6-9be2-4c02-bf52-78d68ce83542 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:15.092964 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c9aa6-9be2-4c02-bf52-78d68ce83542 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:15.093256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c9aa6-9be2-4c02-bf52-78d68ce83542 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:15.094981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 80/156] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:15.100868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4792-7d66-498c-8168-150234936bf4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:15.101378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4792-7d66-498c-8168-150234936bf4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:15.101666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4792-7d66-498c-8168-150234936bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:15.121069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-abca4792-7d66-498c-8168-150234936bf4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:15.122801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-abca4792-7d66-498c-8168-150234936bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:15.161048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 77/157] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:37:14 2022] PUT /identity/v3/projects/ca796ca2de804171942d2631cccd56c7/users/0d2f3620f7b6460ab31d4270adba0f11/roles/3394eca9fdfe4986becd456e050c26e3 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:15.205449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 81/158] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:15.259651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de07f0f-e581-47bb-8ec8-15d7e3fd87c1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:15.355509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:15.388970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de07f0f-e581-47bb-8ec8-15d7e3fd87c1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:15.426422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e13aec-5409-48af-9bbf-cc6769e3b149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:15.427249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e13aec-5409-48af-9bbf-cc6769e3b149 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:15.427529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e13aec-5409-48af-9bbf-cc6769e3b149 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:15.427806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e13aec-5409-48af-9bbf-cc6769e3b149 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:15.441814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e13aec-5409-48af-9bbf-cc6769e3b149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d9e13aec-5409-48af-9bbf-cc6769e3b149 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:15.455057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:15.455990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:15.456804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 78/159] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:15.466560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4792-7d66-498c-8168-150234936bf4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:15.543229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae6554-a473-4ed2-8aaa-2db51ab8e40f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:15.543737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae6554-a473-4ed2-8aaa-2db51ab8e40f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:15.543816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae6554-a473-4ed2-8aaa-2db51ab8e40f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:15.545396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 79/160] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:15.551321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e31d55-1d82-4221-b408-dbdf2b300857 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:15.551607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e31d55-1d82-4221-b408-dbdf2b300857 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:15.551891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e31d55-1d82-4221-b408-dbdf2b300857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:15.569031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-58e31d55-1d82-4221-b408-dbdf2b300857 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:15.570559 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-58e31d55-1d82-4221-b408-dbdf2b300857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:15.613855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4792-7d66-498c-8168-150234936bf4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:15.646412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:15.653714 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6509f-f9f3-4043-93c3-6707fb943740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:15.654541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6509f-f9f3-4043-93c3-6707fb943740 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:15.654933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6509f-f9f3-4043-93c3-6707fb943740 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:15.655329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6509f-f9f3-4043-93c3-6707fb943740 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:15.656182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6509f-f9f3-4043-93c3-6707fb943740 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:15.656589 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6509f-f9f3-4043-93c3-6707fb943740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:15.658313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6509f-f9f3-4043-93c3-6707fb943740 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:15.668233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 82/161] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:15.675640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6509f-f9f3-4043-93c3-6707fb943740 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:15.737079 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 80/162] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:15.831646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6509f-f9f3-4043-93c3-6707fb943740 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:15.843604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e31d55-1d82-4221-b408-dbdf2b300857 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:15.895148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a73bc-bf23-4818-8f58-6b216c8ce778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:15.896552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a73bc-bf23-4818-8f58-6b216c8ce778 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:15.896931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a73bc-bf23-4818-8f58-6b216c8ce778 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:15.897300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a73bc-bf23-4818-8f58-6b216c8ce778 None admin] PATH_INFO: `/v3/users/c987d6e5675d41e7b4123050fafb4d99` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:15.899990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a73bc-bf23-4818-8f58-6b216c8ce778 None admin] RBAC: Authorizing `identity:get_user(user_id=c987d6e5675d41e7b4123050fafb4d99)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:15.901726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a73bc-bf23-4818-8f58-6b216c8ce778 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:15.903823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 83/163] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:15 2022] GET /identity/v3/users/c987d6e5675d41e7b4123050fafb4d99 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:15.905018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0193b14-d2a4-41e1-a06d-576dace9c0eb None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:15.905394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0193b14-d2a4-41e1-a06d-576dace9c0eb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:15.905786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0193b14-d2a4-41e1-a06d-576dace9c0eb None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:15.907257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 84/164] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:15.912942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0193b14-d2a4-41e1-a06d-576dace9c0eb None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:15.914946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:15.963639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e31d55-1d82-4221-b408-dbdf2b300857 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:16.019438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36695e-d9af-4ec5-9071-dbcbd4a2c225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:16.020233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36695e-d9af-4ec5-9071-dbcbd4a2c225 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:16.020514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36695e-d9af-4ec5-9071-dbcbd4a2c225 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:16.020788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36695e-d9af-4ec5-9071-dbcbd4a2c225 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:16.028526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:16.039396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36695e-d9af-4ec5-9071-dbcbd4a2c225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-5b36695e-d9af-4ec5-9071-dbcbd4a2c225 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:16.048006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:16.048991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:16.049769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 81/165] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:16.051721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb32c9-d956-4453-b057-d500cb047ed2 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:16.053032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb32c9-d956-4453-b057-d500cb047ed2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:16.056097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb32c9-d956-4453-b057-d500cb047ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:16.092687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-73bb32c9-d956-4453-b057-d500cb047ed2 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:16.093865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-73bb32c9-d956-4453-b057-d500cb047ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:16.107758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0193b14-d2a4-41e1-a06d-576dace9c0eb None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:16.160610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d902157-a92c-42ea-8489-827136fdf37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:16.161174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d902157-a92c-42ea-8489-827136fdf37d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:16.161454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d902157-a92c-42ea-8489-827136fdf37d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:16.161726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d902157-a92c-42ea-8489-827136fdf37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:16.219325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 82/166] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:16.220175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb32c9-d956-4453-b057-d500cb047ed2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:16.274712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:16.318719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d902157-a92c-42ea-8489-827136fdf37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:16.320234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d902157-a92c-42ea-8489-827136fdf37d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:16.326561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb32c9-d956-4453-b057-d500cb047ed2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:16.396419 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831518d6-b7cc-4772-9bc4-3317595db8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:16.397488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831518d6-b7cc-4772-9bc4-3317595db8f8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:16.398001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831518d6-b7cc-4772-9bc4-3317595db8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:16.398493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831518d6-b7cc-4772-9bc4-3317595db8f8 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:16.399532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831518d6-b7cc-4772-9bc4-3317595db8f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:16.400052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831518d6-b7cc-4772-9bc4-3317595db8f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:16.401650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831518d6-b7cc-4772-9bc4-3317595db8f8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:16.421472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 83/167] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:16.422634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831518d6-b7cc-4772-9bc4-3317595db8f8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:16.467710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:16.580702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 85/168] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:16.581642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d902157-a92c-42ea-8489-827136fdf37d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:16.641621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831518d6-b7cc-4772-9bc4-3317595db8f8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:16.665558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:16.719911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd4592-6ccf-4a6e-9e5e-e2f2823cbc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:16.720734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd4592-6ccf-4a6e-9e5e-e2f2823cbc21 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:16.721022 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd4592-6ccf-4a6e-9e5e-e2f2823cbc21 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:16.721303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd4592-6ccf-4a6e-9e5e-e2f2823cbc21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:16.736387 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd4592-6ccf-4a6e-9e5e-e2f2823cbc21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-0ccd4592-6ccf-4a6e-9e5e-e2f2823cbc21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:16.744762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:16.745678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:16.746454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 84/169] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:16.747083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd4592-6ccf-4a6e-9e5e-e2f2823cbc21 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:16.820176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d902157-a92c-42ea-8489-827136fdf37d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:16.881701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fec0a9-3661-458f-822e-d164597ca29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:16.882451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fec0a9-3661-458f-822e-d164597ca29e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:16.882737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fec0a9-3661-458f-822e-d164597ca29e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:16.883013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fec0a9-3661-458f-822e-d164597ca29e None admin] PATH_INFO: `/v3/users/c987d6e5675d41e7b4123050fafb4d99` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:16.886947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fec0a9-3661-458f-822e-d164597ca29e None admin] RBAC: Authorizing `identity:get_user(user_id=c987d6e5675d41e7b4123050fafb4d99)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:16.888420 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fec0a9-3661-458f-822e-d164597ca29e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:16.890072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 86/170] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:16 2022] GET /identity/v3/users/c987d6e5675d41e7b4123050fafb4d99 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:16.890927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fec0a9-3661-458f-822e-d164597ca29e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:16.912615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd4592-6ccf-4a6e-9e5e-e2f2823cbc21 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:16.926528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:16.983997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f259c8-bdf9-4903-a8fd-57b0a45a1cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:16.985344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f259c8-bdf9-4903-a8fd-57b0a45a1cb8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:16.986520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f259c8-bdf9-4903-a8fd-57b0a45a1cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:16.987146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f259c8-bdf9-4903-a8fd-57b0a45a1cb8 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:16.990466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f259c8-bdf9-4903-a8fd-57b0a45a1cb8 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:16.993060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f259c8-bdf9-4903-a8fd-57b0a45a1cb8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:16.997412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 85/171] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:16 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:16.999401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f259c8-bdf9-4903-a8fd-57b0a45a1cb8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:17.038059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fec0a9-3661-458f-822e-d164597ca29e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:17.093265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:17.121364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e9e98-2d43-4cf3-a454-ee8a28408c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:17.122071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e9e98-2d43-4cf3-a454-ee8a28408c97 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:17.122350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e9e98-2d43-4cf3-a454-ee8a28408c97 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:17.122621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e9e98-2d43-4cf3-a454-ee8a28408c97 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:17.123481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e9e98-2d43-4cf3-a454-ee8a28408c97 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:17.123786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e9e98-2d43-4cf3-a454-ee8a28408c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:17.125230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e9e98-2d43-4cf3-a454-ee8a28408c97 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:17.147228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 87/172] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:17.148086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e9e98-2d43-4cf3-a454-ee8a28408c97 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:17.223380 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f259c8-bdf9-4903-a8fd-57b0a45a1cb8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:17.307651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13d0bd-3907-4e04-9829-71cad765d674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:17.308310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13d0bd-3907-4e04-9829-71cad765d674 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:17.308593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13d0bd-3907-4e04-9829-71cad765d674 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:17.308865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13d0bd-3907-4e04-9829-71cad765d674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:17.362007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:17.393557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e9e98-2d43-4cf3-a454-ee8a28408c97 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:17.451244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13d0bd-3907-4e04-9829-71cad765d674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:17.452974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13d0bd-3907-4e04-9829-71cad765d674 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:17.486630 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50b1d0-5622-4851-95ef-61fc461398c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:17.487310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50b1d0-5622-4851-95ef-61fc461398c9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:17.487601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50b1d0-5622-4851-95ef-61fc461398c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:17.487885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50b1d0-5622-4851-95ef-61fc461398c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:17.488738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50b1d0-5622-4851-95ef-61fc461398c9 None admin] RBAC: Adding query filter params (role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=ca796ca2de804171942d2631cccd56c7, user.id=c987d6e5675d41e7b4123050fafb4d99) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:17.489051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50b1d0-5622-4851-95ef-61fc461398c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:17.490613 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50b1d0-5622-4851-95ef-61fc461398c9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:17.501193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 88/173] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:17 2022] GET /identity/v3/role_assignments?user.id=c987d6e5675d41e7b4123050fafb4d99&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=7583ad887d8b4052bd11783371c338b4 => generated 608 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:17.502015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50b1d0-5622-4851-95ef-61fc461398c9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:17.531617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:17.662981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 86/174] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:17.669429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13d0bd-3907-4e04-9829-71cad765d674 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:17.752040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50b1d0-5622-4851-95ef-61fc461398c9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:17.819048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13d0bd-3907-4e04-9829-71cad765d674 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:17.858607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e2272-c379-447f-b596-4c6dc9ba0b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:17.860149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e2272-c379-447f-b596-4c6dc9ba0b20 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:17.860588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e2272-c379-447f-b596-4c6dc9ba0b20 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:17.861020 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e2272-c379-447f-b596-4c6dc9ba0b20 None admin] PATH_INFO: `/v3/users/c987d6e5675d41e7b4123050fafb4d99` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:17.862622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e2272-c379-447f-b596-4c6dc9ba0b20 None admin] RBAC: Authorizing `identity:get_user(user_id=c987d6e5675d41e7b4123050fafb4d99)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:17.864310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e2272-c379-447f-b596-4c6dc9ba0b20 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:17.866159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 89/175] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:17 2022] GET /identity/v3/users/c987d6e5675d41e7b4123050fafb4d99 => generated 275 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:17.872139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e2272-c379-447f-b596-4c6dc9ba0b20 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:17.873251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db904283-2615-4d4d-8573-eaeb11cde61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:17.874147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db904283-2615-4d4d-8573-eaeb11cde61b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:17.874431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db904283-2615-4d4d-8573-eaeb11cde61b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:17.874706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db904283-2615-4d4d-8573-eaeb11cde61b None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:17.878485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db904283-2615-4d4d-8573-eaeb11cde61b None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:17.879978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db904283-2615-4d4d-8573-eaeb11cde61b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:17.882058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 87/176] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:17 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:17.891139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db904283-2615-4d4d-8573-eaeb11cde61b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:17.907437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:17.908562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:18.033376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e2272-c379-447f-b596-4c6dc9ba0b20 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:18.088602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db904283-2615-4d4d-8573-eaeb11cde61b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:18.099317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c13288-f4de-4942-bff0-b1ba0ca6f62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:18.099995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c13288-f4de-4942-bff0-b1ba0ca6f62d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.100324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c13288-f4de-4942-bff0-b1ba0ca6f62d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.100594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c13288-f4de-4942-bff0-b1ba0ca6f62d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.145399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c4f9b-b7a7-4ee1-afbf-1693ea09590e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:18.146279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c4f9b-b7a7-4ee1-afbf-1693ea09590e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.146921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c4f9b-b7a7-4ee1-afbf-1693ea09590e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.147346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c4f9b-b7a7-4ee1-afbf-1693ea09590e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.148312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c4f9b-b7a7-4ee1-afbf-1693ea09590e None admin] RBAC: Adding query filter params (role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=ca796ca2de804171942d2631cccd56c7, user.id=c987d6e5675d41e7b4123050fafb4d99) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:18.148783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c4f9b-b7a7-4ee1-afbf-1693ea09590e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:18.151368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c4f9b-b7a7-4ee1-afbf-1693ea09590e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:18.159021 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 88/177] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:17 2022] GET /identity/v3/role_assignments?user.id=c987d6e5675d41e7b4123050fafb4d99&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:18.160213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d49271-f03f-43d4-976b-e3278dfdea7d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.160894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d49271-f03f-43d4-976b-e3278dfdea7d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.161172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d49271-f03f-43d4-976b-e3278dfdea7d None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.162520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 89/178] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:18.163988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bb915-abfd-446a-b035-27131cf6aefe None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.164630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bb915-abfd-446a-b035-27131cf6aefe None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.165008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bb915-abfd-446a-b035-27131cf6aefe None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.167013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 90/179] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:18.172579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76020e84-3aa6-4825-890e-94afff669ee8 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.173049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76020e84-3aa6-4825-890e-94afff669ee8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.173333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76020e84-3aa6-4825-890e-94afff669ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.193271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-76020e84-3aa6-4825-890e-94afff669ee8 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:18.194537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-76020e84-3aa6-4825-890e-94afff669ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:18.203517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c13288-f4de-4942-bff0-b1ba0ca6f62d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:18.205063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c13288-f4de-4942-bff0-b1ba0ca6f62d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:18.230368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:18.287351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 91/180] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:18.288721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df1a8a-07fb-4064-af89-280cdf96f334 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.289010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df1a8a-07fb-4064-af89-280cdf96f334 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.289290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df1a8a-07fb-4064-af89-280cdf96f334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.304023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:18.325449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-54df1a8a-07fb-4064-af89-280cdf96f334 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:18.326591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-54df1a8a-07fb-4064-af89-280cdf96f334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:18.339086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 90/181] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 09:37:18.349106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c13288-f4de-4942-bff0-b1ba0ca6f62d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:18.426229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 92/182] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:18.426690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df1a8a-07fb-4064-af89-280cdf96f334 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:18.470298 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c13288-f4de-4942-bff0-b1ba0ca6f62d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:18.493935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:18.509811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15999250-dfea-4742-92af-f7be85f759df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:18.510849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15999250-dfea-4742-92af-f7be85f759df None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.511235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15999250-dfea-4742-92af-f7be85f759df None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.513905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15999250-dfea-4742-92af-f7be85f759df None admin] PATH_INFO: `/v3/projects/5df1b615b14642ffb284dbfab9a1fee2` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.513905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15999250-dfea-4742-92af-f7be85f759df None admin] RBAC: Authorizing `identity:get_project(project_id=5df1b615b14642ffb284dbfab9a1fee2)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:18.514645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15999250-dfea-4742-92af-f7be85f759df None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:18.516951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 91/183] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:18 2022] GET /identity/v3/projects/5df1b615b14642ffb284dbfab9a1fee2 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:18.517811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15999250-dfea-4742-92af-f7be85f759df None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:18.616009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df1a8a-07fb-4064-af89-280cdf96f334 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:18.652872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15999250-dfea-4742-92af-f7be85f759df None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:18.657155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:18.667060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d36ee-bc30-4b1d-b0a4-f0b34304694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:18.667990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d36ee-bc30-4b1d-b0a4-f0b34304694c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.668275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d36ee-bc30-4b1d-b0a4-f0b34304694c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.668616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d36ee-bc30-4b1d-b0a4-f0b34304694c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.679217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d36ee-bc30-4b1d-b0a4-f0b34304694c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ad4d36ee-bc30-4b1d-b0a4-f0b34304694c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:18.687161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:18.688004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:18.689366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 93/184] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:37:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:18.690449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d36ee-bc30-4b1d-b0a4-f0b34304694c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:18.694904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2bee1-b363-4f83-853b-3675c2b82609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:18.695720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2bee1-b363-4f83-853b-3675c2b82609 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.695998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2bee1-b363-4f83-853b-3675c2b82609 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.696802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2bee1-b363-4f83-853b-3675c2b82609 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.707500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2bee1-b363-4f83-853b-3675c2b82609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-2bc2bee1-b363-4f83-853b-3675c2b82609 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:18.716270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:18.717103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:18.717869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 92/185] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:18.723514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec13108-d696-4343-956d-15c4023cc6f6 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.723514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec13108-d696-4343-956d-15c4023cc6f6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.723514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec13108-d696-4343-956d-15c4023cc6f6 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.723514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 93/186] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:18.723514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec13108-d696-4343-956d-15c4023cc6f6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:18.769404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:18.799875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d36ee-bc30-4b1d-b0a4-f0b34304694c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:18.821782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec13108-d696-4343-956d-15c4023cc6f6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:18.833310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ea010-2d59-4c0d-8cb8-54ccd862c9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:18.834050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ea010-2d59-4c0d-8cb8-54ccd862c9c2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.834331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ea010-2d59-4c0d-8cb8-54ccd862c9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.834604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ea010-2d59-4c0d-8cb8-54ccd862c9c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.835424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ea010-2d59-4c0d-8cb8-54ccd862c9c2 None admin] RBAC: Adding query filter params (role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=5df1b615b14642ffb284dbfab9a1fee2, user.id=c987d6e5675d41e7b4123050fafb4d99) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:18.835730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ea010-2d59-4c0d-8cb8-54ccd862c9c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:18.837743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ea010-2d59-4c0d-8cb8-54ccd862c9c2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:18.843800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 94/187] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:18 2022] GET /identity/v3/role_assignments?user.id=c987d6e5675d41e7b4123050fafb4d99&scope.project.id=5df1b615b14642ffb284dbfab9a1fee2&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:18.845304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6fa35-4ada-4753-8d3d-3dba77dd49be None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.845592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6fa35-4ada-4753-8d3d-3dba77dd49be None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.845867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6fa35-4ada-4753-8d3d-3dba77dd49be None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.847266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 95/188] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:18.847901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6fa35-4ada-4753-8d3d-3dba77dd49be None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:18.859466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427cba5-43c6-429b-8414-2074b6616cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:18.860164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427cba5-43c6-429b-8414-2074b6616cc0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.860449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427cba5-43c6-429b-8414-2074b6616cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.860735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427cba5-43c6-429b-8414-2074b6616cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.861515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427cba5-43c6-429b-8414-2074b6616cc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:18.861826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427cba5-43c6-429b-8414-2074b6616cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:18.863392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427cba5-43c6-429b-8414-2074b6616cc0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:18.874801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 94/189] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:18.880756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dafdd8-8a50-49d1-97ea-929dab9c2f29 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.881052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dafdd8-8a50-49d1-97ea-929dab9c2f29 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.881338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dafdd8-8a50-49d1-97ea-929dab9c2f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:18.901158 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-52dafdd8-8a50-49d1-97ea-929dab9c2f29 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:18.902439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-52dafdd8-8a50-49d1-97ea-929dab9c2f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:18.915234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:18.963567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6fa35-4ada-4753-8d3d-3dba77dd49be None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:18.980503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 95/190] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:18.987505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd41b7-8455-41bf-b32b-bf90d1950328 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:18.988030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd41b7-8455-41bf-b32b-bf90d1950328 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:18.988416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd41b7-8455-41bf-b32b-bf90d1950328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.003457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ea475-f3b4-419e-bf8b-f09f425d0ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:19.004625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ea475-f3b4-419e-bf8b-f09f425d0ba0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:19.005002 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ea475-f3b4-419e-bf8b-f09f425d0ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:19.005372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ea475-f3b4-419e-bf8b-f09f425d0ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.006327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ea475-f3b4-419e-bf8b-f09f425d0ba0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:19.006735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ea475-f3b4-419e-bf8b-f09f425d0ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:19.008261 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ea475-f3b4-419e-bf8b-f09f425d0ba0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:19.012070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-87dd41b7-8455-41bf-b32b-bf90d1950328 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:19.013309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-87dd41b7-8455-41bf-b32b-bf90d1950328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:19.018208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 96/191] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:19.023738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ea475-f3b4-419e-bf8b-f09f425d0ba0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:19.056221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:19.095907 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 96/192] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:19.096762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd41b7-8455-41bf-b32b-bf90d1950328 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:19.142203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ea475-f3b4-419e-bf8b-f09f425d0ba0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:19.158798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:19.175436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcc738-ab62-4e78-bcc4-4bdd8326be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:19.176306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcc738-ab62-4e78-bcc4-4bdd8326be95 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:19.176590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcc738-ab62-4e78-bcc4-4bdd8326be95 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:19.176868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcc738-ab62-4e78-bcc4-4bdd8326be95 None admin] PATH_INFO: `/v3/users/408be52fb987442f88fc591ee9126f2f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.189622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcc738-ab62-4e78-bcc4-4bdd8326be95 None admin] RBAC: Authorizing `identity:get_user(user_id=408be52fb987442f88fc591ee9126f2f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:19.191110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcc738-ab62-4e78-bcc4-4bdd8326be95 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:19.192921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 97/193] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:19 2022] GET /identity/v3/users/408be52fb987442f88fc591ee9126f2f => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:19.193600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcc738-ab62-4e78-bcc4-4bdd8326be95 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:19.198642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd41b7-8455-41bf-b32b-bf90d1950328 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:19.232183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a116643-7449-4461-8bdd-04d1dc9771d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:19.232967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a116643-7449-4461-8bdd-04d1dc9771d4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:19.233254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a116643-7449-4461-8bdd-04d1dc9771d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:19.233534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a116643-7449-4461-8bdd-04d1dc9771d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.234720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a116643-7449-4461-8bdd-04d1dc9771d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:19.236171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a116643-7449-4461-8bdd-04d1dc9771d4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:19.237769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 97/194] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:19.238559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a116643-7449-4461-8bdd-04d1dc9771d4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:19.277005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:19.293074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcc738-ab62-4e78-bcc4-4bdd8326be95 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:19.326649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d551ce-a258-4b82-bb33-60eabd072a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:19.327510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d551ce-a258-4b82-bb33-60eabd072a16 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:19.327788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d551ce-a258-4b82-bb33-60eabd072a16 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:19.328061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d551ce-a258-4b82-bb33-60eabd072a16 None admin] PATH_INFO: `/v3/users/408be52fb987442f88fc591ee9126f2f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.329220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d551ce-a258-4b82-bb33-60eabd072a16 None admin] RBAC: Authorizing `identity:get_user(user_id=408be52fb987442f88fc591ee9126f2f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:19.330683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d551ce-a258-4b82-bb33-60eabd072a16 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:19.332301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 98/195] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:19 2022] GET /identity/v3/users/408be52fb987442f88fc591ee9126f2f => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:19.337451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d551ce-a258-4b82-bb33-60eabd072a16 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:19.340429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a116643-7449-4461-8bdd-04d1dc9771d4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:19.373601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b79d81-86ff-4258-b69a-3c370a396558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:19.374416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b79d81-86ff-4258-b69a-3c370a396558 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:19.374699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b79d81-86ff-4258-b69a-3c370a396558 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:19.374974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b79d81-86ff-4258-b69a-3c370a396558 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.376838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:19.385375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b79d81-86ff-4258-b69a-3c370a396558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-35b79d81-86ff-4258-b69a-3c370a396558 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:19.393164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:19.394188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:19.395167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:19.398018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 98/196] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:19.398929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b79d81-86ff-4258-b69a-3c370a396558 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:19.430348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d551ce-a258-4b82-bb33-60eabd072a16 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:19.464632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9cd2e-ae6c-4f13-84a6-214fd3b3e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:19.465274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9cd2e-ae6c-4f13-84a6-214fd3b3e167 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:19.465557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9cd2e-ae6c-4f13-84a6-214fd3b3e167 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:19.465834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9cd2e-ae6c-4f13-84a6-214fd3b3e167 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.497846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:19.501284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b79d81-86ff-4258-b69a-3c370a396558 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:19.542671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46d7c5-7bfa-48fe-b971-7c71ae261522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:19.543333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46d7c5-7bfa-48fe-b971-7c71ae261522 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:19.543621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46d7c5-7bfa-48fe-b971-7c71ae261522 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:19.543818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9cd2e-ae6c-4f13-84a6-214fd3b3e167 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:19.543910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46d7c5-7bfa-48fe-b971-7c71ae261522 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.544605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46d7c5-7bfa-48fe-b971-7c71ae261522 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:19.545293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9cd2e-ae6c-4f13-84a6-214fd3b3e167 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:19.546074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46d7c5-7bfa-48fe-b971-7c71ae261522 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:19.588505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:19.642873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 99/197] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:19.654188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 99/198] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:19 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:19.654395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9cd2e-ae6c-4f13-84a6-214fd3b3e167 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:19.654486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46d7c5-7bfa-48fe-b971-7c71ae261522 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:19.756609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9cd2e-ae6c-4f13-84a6-214fd3b3e167 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:19.786710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46d7c5-7bfa-48fe-b971-7c71ae261522 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:19.796280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caca8b-5688-4ac1-b5d8-d750a623f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:19.796952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caca8b-5688-4ac1-b5d8-d750a623f053 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:19.797242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caca8b-5688-4ac1-b5d8-d750a623f053 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:19.797517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caca8b-5688-4ac1-b5d8-d750a623f053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.814990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:19.825923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bac652-9eab-4699-a270-0e56117bc57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:19.827756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bac652-9eab-4699-a270-0e56117bc57d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:19.828096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bac652-9eab-4699-a270-0e56117bc57d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:19.828385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bac652-9eab-4699-a270-0e56117bc57d None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:19.829149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bac652-9eab-4699-a270-0e56117bc57d None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:19.829660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bac652-9eab-4699-a270-0e56117bc57d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:19.830981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bac652-9eab-4699-a270-0e56117bc57d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:19.842233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 100/199] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:19.843326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bac652-9eab-4699-a270-0e56117bc57d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:19.865591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:19.882545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9caca8b-5688-4ac1-b5d8-d750a623f053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:19.884052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9caca8b-5688-4ac1-b5d8-d750a623f053 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:19.957735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bac652-9eab-4699-a270-0e56117bc57d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:20.003968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422a1ed-918d-400b-9278-b1331ade8023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:20.004992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422a1ed-918d-400b-9278-b1331ade8023 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:20.005281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422a1ed-918d-400b-9278-b1331ade8023 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:20.005578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422a1ed-918d-400b-9278-b1331ade8023 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:20.007575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422a1ed-918d-400b-9278-b1331ade8023 None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:20.008362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422a1ed-918d-400b-9278-b1331ade8023 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:20.011056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 101/200] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:19 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:20.011655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422a1ed-918d-400b-9278-b1331ade8023 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:20.014056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 100/201] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:20.016765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caca8b-5688-4ac1-b5d8-d750a623f053 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:20.069125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:20.136794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caca8b-5688-4ac1-b5d8-d750a623f053 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:20.137686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:20.162264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422a1ed-918d-400b-9278-b1331ade8023 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:20.184255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4872a-1590-45aa-a325-c8f57bc7960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:20.185407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4872a-1590-45aa-a325-c8f57bc7960e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:20.185834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4872a-1590-45aa-a325-c8f57bc7960e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:20.186324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4872a-1590-45aa-a325-c8f57bc7960e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:20.187310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4872a-1590-45aa-a325-c8f57bc7960e None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=a91190fa89bd41f49048e6128973a986, user.id=408be52fb987442f88fc591ee9126f2f) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:20.187645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4872a-1590-45aa-a325-c8f57bc7960e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:20.189755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4872a-1590-45aa-a325-c8f57bc7960e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:20.197723 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 101/202] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:20 2022] GET /identity/v3/role_assignments?user.id=408be52fb987442f88fc591ee9126f2f&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=fe4d636f5c4842f18569347911e6cdec => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:20.198393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4872a-1590-45aa-a325-c8f57bc7960e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:20.200114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75374399-f2c7-4394-a5ae-14a2cdb1075c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:20.201140 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75374399-f2c7-4394-a5ae-14a2cdb1075c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:20.201577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75374399-f2c7-4394-a5ae-14a2cdb1075c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:20.201970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75374399-f2c7-4394-a5ae-14a2cdb1075c None admin] PATH_INFO: `/v3/users/0d2f3620f7b6460ab31d4270adba0f11` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:20.203308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75374399-f2c7-4394-a5ae-14a2cdb1075c None admin] RBAC: Authorizing `identity:get_user(user_id=0d2f3620f7b6460ab31d4270adba0f11)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:20.204831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75374399-f2c7-4394-a5ae-14a2cdb1075c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:20.206783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 102/203] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:20 2022] GET /identity/v3/users/0d2f3620f7b6460ab31d4270adba0f11 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:20.212955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75374399-f2c7-4394-a5ae-14a2cdb1075c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:20.327965 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4872a-1590-45aa-a325-c8f57bc7960e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:20.344284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:20.392249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45519b8d-71d8-4a20-a0ec-9524950bebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:20.393145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45519b8d-71d8-4a20-a0ec-9524950bebed None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:20.393433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45519b8d-71d8-4a20-a0ec-9524950bebed None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:20.393714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45519b8d-71d8-4a20-a0ec-9524950bebed None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:20.398738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45519b8d-71d8-4a20-a0ec-9524950bebed None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:20.400757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45519b8d-71d8-4a20-a0ec-9524950bebed None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:20.403723 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 102/204] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:20 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:20.408970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45519b8d-71d8-4a20-a0ec-9524950bebed None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:20.415845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75374399-f2c7-4394-a5ae-14a2cdb1075c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:20.453729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ca445-93e3-4d17-a8ae-8fad15f62256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:20.454491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ca445-93e3-4d17-a8ae-8fad15f62256 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:20.454973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ca445-93e3-4d17-a8ae-8fad15f62256 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:20.455096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ca445-93e3-4d17-a8ae-8fad15f62256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:20.479306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:20.528445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ca445-93e3-4d17-a8ae-8fad15f62256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:20.530330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ca445-93e3-4d17-a8ae-8fad15f62256 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:20.631426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45519b8d-71d8-4a20-a0ec-9524950bebed None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:20.632305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 103/205] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:20.639139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ca445-93e3-4d17-a8ae-8fad15f62256 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:20.693250 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402aa7c-6771-4519-aa7c-2397b3be087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:20.693947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402aa7c-6771-4519-aa7c-2397b3be087d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:20.694235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402aa7c-6771-4519-aa7c-2397b3be087d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:20.694515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402aa7c-6771-4519-aa7c-2397b3be087d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:20.695354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402aa7c-6771-4519-aa7c-2397b3be087d None admin] RBAC: Adding query filter params (role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=a91190fa89bd41f49048e6128973a986, user.id=408be52fb987442f88fc591ee9126f2f) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:20.695664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402aa7c-6771-4519-aa7c-2397b3be087d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:20.697652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402aa7c-6771-4519-aa7c-2397b3be087d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:20.716075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 103/206] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:20 2022] GET /identity/v3/role_assignments?user.id=408be52fb987442f88fc591ee9126f2f&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=7583ad887d8b4052bd11783371c338b4 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:20.734414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:20.792275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ca445-93e3-4d17-a8ae-8fad15f62256 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:20.831376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d567d2-4970-4377-9470-21d5f45b29c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:20.832282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d567d2-4970-4377-9470-21d5f45b29c8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:20.832553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d567d2-4970-4377-9470-21d5f45b29c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:20.832824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d567d2-4970-4377-9470-21d5f45b29c8 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:20.834107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d567d2-4970-4377-9470-21d5f45b29c8 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:20.835607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d567d2-4970-4377-9470-21d5f45b29c8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:20.838139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 104/207] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:20 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:20.843973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402aa7c-6771-4519-aa7c-2397b3be087d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:20.890262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:20.965836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402aa7c-6771-4519-aa7c-2397b3be087d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:21.002268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4ff54-20d2-4c9e-90ad-003b4423d305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:21.002856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4ff54-20d2-4c9e-90ad-003b4423d305 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:21.003142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4ff54-20d2-4c9e-90ad-003b4423d305 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:21.003426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4ff54-20d2-4c9e-90ad-003b4423d305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:21.004276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4ff54-20d2-4c9e-90ad-003b4423d305 None admin] RBAC: Adding query filter params (role.id=3394eca9fdfe4986becd456e050c26e3, scope.project.id=ca796ca2de804171942d2631cccd56c7, user.id=0d2f3620f7b6460ab31d4270adba0f11) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:21.004594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4ff54-20d2-4c9e-90ad-003b4423d305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:21.006169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4ff54-20d2-4c9e-90ad-003b4423d305 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:21.014693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 104/208] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:20 2022] GET /identity/v3/role_assignments?user.id=0d2f3620f7b6460ab31d4270adba0f11&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=3394eca9fdfe4986becd456e050c26e3 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:22.345278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d56f4-f5ab-4ace-a2c1-f202e3fb0ffc None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.345789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d56f4-f5ab-4ace-a2c1-f202e3fb0ffc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.345912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d56f4-f5ab-4ace-a2c1-f202e3fb0ffc None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.347229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 105/209] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:22.353048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d165a9-22e3-46d6-8363-7f5697dba3a4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.353435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d165a9-22e3-46d6-8363-7f5697dba3a4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.353615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d165a9-22e3-46d6-8363-7f5697dba3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.370280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-25d165a9-22e3-46d6-8363-7f5697dba3a4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:22.371473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-25d165a9-22e3-46d6-8363-7f5697dba3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:22.386180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2de634-20fc-4673-bbbe-fcf3a0f091ab None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.386464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2de634-20fc-4673-bbbe-fcf3a0f091ab None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.386744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2de634-20fc-4673-bbbe-fcf3a0f091ab None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.388106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 106/210] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:22.394808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8665f-b715-49cb-b699-a9da90d9d8a3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.395090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8665f-b715-49cb-b699-a9da90d9d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.395362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8665f-b715-49cb-b699-a9da90d9d8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.412901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3ee8665f-b715-49cb-b699-a9da90d9d8a3 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:22.415438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3ee8665f-b715-49cb-b699-a9da90d9d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:22.450417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 105/211] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:22.474302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d165a9-22e3-46d6-8363-7f5697dba3a4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:22.499342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 107/212] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:22.511004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:22.525158 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8665f-b715-49cb-b699-a9da90d9d8a3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:22.578303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d165a9-22e3-46d6-8363-7f5697dba3a4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:22.578496 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:22.613302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365ade4-06db-48dd-923a-88142e7325f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:22.614069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365ade4-06db-48dd-923a-88142e7325f2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.614354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365ade4-06db-48dd-923a-88142e7325f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.614629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365ade4-06db-48dd-923a-88142e7325f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.615799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365ade4-06db-48dd-923a-88142e7325f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:22.617237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365ade4-06db-48dd-923a-88142e7325f2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:22.619466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 106/213] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:22.625152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365ade4-06db-48dd-923a-88142e7325f2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:22.633160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8665f-b715-49cb-b699-a9da90d9d8a3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:22.670864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e09a0-3f33-4ff1-8f71-0db7454d2ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:22.671661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e09a0-3f33-4ff1-8f71-0db7454d2ab0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.671949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e09a0-3f33-4ff1-8f71-0db7454d2ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.672225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e09a0-3f33-4ff1-8f71-0db7454d2ab0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.682432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e09a0-3f33-4ff1-8f71-0db7454d2ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-fb1e09a0-3f33-4ff1-8f71-0db7454d2ab0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:22.690143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:22.691022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 108/214] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:22.696643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff6512-00a4-4688-9add-8ae541eb1797 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.697351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff6512-00a4-4688-9add-8ae541eb1797 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.697351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff6512-00a4-4688-9add-8ae541eb1797 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.697351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 109/215] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:22.697351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff6512-00a4-4688-9add-8ae541eb1797 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:22.742146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365ade4-06db-48dd-923a-88142e7325f2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:22.744514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:22.777787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6948f4-13ad-4d37-8a0f-2d01912c71d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:22.778499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6948f4-13ad-4d37-8a0f-2d01912c71d8 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.778786 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6948f4-13ad-4d37-8a0f-2d01912c71d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.779073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6948f4-13ad-4d37-8a0f-2d01912c71d8 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.779864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6948f4-13ad-4d37-8a0f-2d01912c71d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:22.781308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6948f4-13ad-4d37-8a0f-2d01912c71d8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:22.806856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff6512-00a4-4688-9add-8ae541eb1797 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:22.817005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:22.839514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19a128-174a-4cce-9e79-ca0230cf56b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:22.840177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19a128-174a-4cce-9e79-ca0230cf56b9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.840462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19a128-174a-4cce-9e79-ca0230cf56b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.840734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19a128-174a-4cce-9e79-ca0230cf56b9 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.841470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19a128-174a-4cce-9e79-ca0230cf56b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:22.841773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19a128-174a-4cce-9e79-ca0230cf56b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:22.843189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19a128-174a-4cce-9e79-ca0230cf56b9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:22.851402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 110/216] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:22.856039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e19748-19c1-4418-a4ae-56f381e2af5f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:22.856039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e19748-19c1-4418-a4ae-56f381e2af5f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:22.856039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e19748-19c1-4418-a4ae-56f381e2af5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:22.870591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-47e19748-19c1-4418-a4ae-56f381e2af5f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:22.871702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-47e19748-19c1-4418-a4ae-56f381e2af5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:22.884464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 107/217] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:22 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:22.885346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6948f4-13ad-4d37-8a0f-2d01912c71d8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:22.974713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 111/218] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:22.998196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e19748-19c1-4418-a4ae-56f381e2af5f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:23.021215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6948f4-13ad-4d37-8a0f-2d01912c71d8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:23.062591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48e971-a423-4ef9-9380-719727cc7c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:23.063550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e971-a423-4ef9-9380-719727cc7c01 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.064238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e971-a423-4ef9-9380-719727cc7c01 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.064238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e971-a423-4ef9-9380-719727cc7c01 None admin] PATH_INFO: `/v3/users/5d98ea05ab1746508ce20c43554bb5e0` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.082877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48e971-a423-4ef9-9380-719727cc7c01 None admin] RBAC: Authorizing `identity:get_user(user_id=5d98ea05ab1746508ce20c43554bb5e0)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:23.084678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48e971-a423-4ef9-9380-719727cc7c01 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:23.087009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 108/219] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:22 2022] GET /identity/v3/users/5d98ea05ab1746508ce20c43554bb5e0 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:23.093077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48e971-a423-4ef9-9380-719727cc7c01 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:23.115978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:23.142355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e19748-19c1-4418-a4ae-56f381e2af5f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:23.147390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:23.182857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2cee-a016-4ce7-83f3-6a0ce4e6341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:23.183675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2cee-a016-4ce7-83f3-6a0ce4e6341b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.183966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2cee-a016-4ce7-83f3-6a0ce4e6341b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.184252 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2cee-a016-4ce7-83f3-6a0ce4e6341b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.198852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c2cee-a016-4ce7-83f3-6a0ce4e6341b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-db9c2cee-a016-4ce7-83f3-6a0ce4e6341b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:23.209351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:23.210273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:23.211028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:23.211706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48e971-a423-4ef9-9380-719727cc7c01 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:23.211792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 112/220] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:23.217664 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2cee-a016-4ce7-83f3-6a0ce4e6341b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:23.260409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cab7b-a797-44cb-84ab-3fe4aac17ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:23.261130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cab7b-a797-44cb-84ab-3fe4aac17ec1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.261437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cab7b-a797-44cb-84ab-3fe4aac17ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.261711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cab7b-a797-44cb-84ab-3fe4aac17ec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.337808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4cab7b-a797-44cb-84ab-3fe4aac17ec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:23.339315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4cab7b-a797-44cb-84ab-3fe4aac17ec1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:23.340085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2cee-a016-4ce7-83f3-6a0ce4e6341b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:23.357674 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:23.390531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a870c-c3d4-4a46-8c29-da1c585beee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:23.392073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a870c-c3d4-4a46-8c29-da1c585beee8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.392588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a870c-c3d4-4a46-8c29-da1c585beee8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.393038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a870c-c3d4-4a46-8c29-da1c585beee8 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.394003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a870c-c3d4-4a46-8c29-da1c585beee8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:23.394903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a870c-c3d4-4a46-8c29-da1c585beee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:23.396878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a870c-c3d4-4a46-8c29-da1c585beee8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:23.409154 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 113/221] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:23.411098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df645-a81a-4fdf-8a71-8bf906218f16 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.411716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df645-a81a-4fdf-8a71-8bf906218f16 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.412941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15df645-a81a-4fdf-8a71-8bf906218f16 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.414743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 114/222] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:23.420253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df645-a81a-4fdf-8a71-8bf906218f16 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:23.448967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 109/223] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:23.450624 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7b013-b1c2-4dcb-90c3-d6be92fd6f1b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.450811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7b013-b1c2-4dcb-90c3-d6be92fd6f1b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.451195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7b013-b1c2-4dcb-90c3-d6be92fd6f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.455111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:23.469316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-12d7b013-b1c2-4dcb-90c3-d6be92fd6f1b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:23.470698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-12d7b013-b1c2-4dcb-90c3-d6be92fd6f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:23.598870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15df645-a81a-4fdf-8a71-8bf906218f16 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:23.617075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 110/224] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:23.617075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7b013-b1c2-4dcb-90c3-d6be92fd6f1b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:23.656564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a7d08-d7ef-462b-a0d8-42cdba16b683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:23.657721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a7d08-d7ef-462b-a0d8-42cdba16b683 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.658232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a7d08-d7ef-462b-a0d8-42cdba16b683 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.658736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a7d08-d7ef-462b-a0d8-42cdba16b683 None admin] PATH_INFO: `/v3/users/408be52fb987442f88fc591ee9126f2f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.660114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a7d08-d7ef-462b-a0d8-42cdba16b683 None admin] RBAC: Authorizing `identity:get_user(user_id=408be52fb987442f88fc591ee9126f2f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:23.662002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a7d08-d7ef-462b-a0d8-42cdba16b683 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:23.663746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 115/225] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:23 2022] GET /identity/v3/users/408be52fb987442f88fc591ee9126f2f => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:23.664764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a7d08-d7ef-462b-a0d8-42cdba16b683 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:23.665946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:23.735272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7b013-b1c2-4dcb-90c3-d6be92fd6f1b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:23.779029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14906168-73a7-4f38-a503-9d2c6f97f7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:23.780617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14906168-73a7-4f38-a503-9d2c6f97f7fc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.781131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14906168-73a7-4f38-a503-9d2c6f97f7fc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.781605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14906168-73a7-4f38-a503-9d2c6f97f7fc None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.783550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14906168-73a7-4f38-a503-9d2c6f97f7fc None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:23.786357 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:23.786574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14906168-73a7-4f38-a503-9d2c6f97f7fc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:23.790178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 111/226] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:23 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:23.791996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14906168-73a7-4f38-a503-9d2c6f97f7fc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:23.797144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a7d08-d7ef-462b-a0d8-42cdba16b683 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:23.838532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41170aa7-95b9-499a-b160-350911de7e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:23.840737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41170aa7-95b9-499a-b160-350911de7e91 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.841276 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41170aa7-95b9-499a-b160-350911de7e91 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.841772 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41170aa7-95b9-499a-b160-350911de7e91 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.856588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41170aa7-95b9-499a-b160-350911de7e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-41170aa7-95b9-499a-b160-350911de7e91 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:23.867000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:23.868011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:23.868775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:23.871467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 116/227] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:23.876853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41170aa7-95b9-499a-b160-350911de7e91 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:23.921699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14906168-73a7-4f38-a503-9d2c6f97f7fc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:23.936933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:23.959118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5370f-9e8f-47e3-b9ed-34a2a29714a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:23.960520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5370f-9e8f-47e3-b9ed-34a2a29714a6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:23.960908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5370f-9e8f-47e3-b9ed-34a2a29714a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:23.961382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5370f-9e8f-47e3-b9ed-34a2a29714a6 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:23.963038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5370f-9e8f-47e3-b9ed-34a2a29714a6 None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:23.964602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5370f-9e8f-47e3-b9ed-34a2a29714a6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:23.971273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 112/228] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:23 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:23.972274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5370f-9e8f-47e3-b9ed-34a2a29714a6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:24.014377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41170aa7-95b9-499a-b160-350911de7e91 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:24.049207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9225576-ef2e-4039-a2aa-82a94d34d668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:24.050475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9225576-ef2e-4039-a2aa-82a94d34d668 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:24.050789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9225576-ef2e-4039-a2aa-82a94d34d668 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:24.051085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9225576-ef2e-4039-a2aa-82a94d34d668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:24.052066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9225576-ef2e-4039-a2aa-82a94d34d668 None admin] RBAC: Adding query filter params (role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=a91190fa89bd41f49048e6128973a986, user.id=5d98ea05ab1746508ce20c43554bb5e0) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:24.052480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9225576-ef2e-4039-a2aa-82a94d34d668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:24.054137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9225576-ef2e-4039-a2aa-82a94d34d668 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:24.061688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 117/229] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:23 2022] GET /identity/v3/role_assignments?user.id=5d98ea05ab1746508ce20c43554bb5e0&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:24.062593 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9225576-ef2e-4039-a2aa-82a94d34d668 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:24.086191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:24.095980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5370f-9e8f-47e3-b9ed-34a2a29714a6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:24.140774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ac2ef-98a2-4aa4-952c-97dcfea7eb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:24.141537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ac2ef-98a2-4aa4-952c-97dcfea7eb53 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:24.141748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ac2ef-98a2-4aa4-952c-97dcfea7eb53 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:24.142068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ac2ef-98a2-4aa4-952c-97dcfea7eb53 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:24.142839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ac2ef-98a2-4aa4-952c-97dcfea7eb53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:24.143147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ac2ef-98a2-4aa4-952c-97dcfea7eb53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:24.144584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ac2ef-98a2-4aa4-952c-97dcfea7eb53 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:24.158385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 113/230] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:24.160223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ac2ef-98a2-4aa4-952c-97dcfea7eb53 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:24.192973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9225576-ef2e-4039-a2aa-82a94d34d668 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:24.209774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:24.242324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f646a-ad2a-4698-8364-63d3b68aa4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:24.244312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f646a-ad2a-4698-8364-63d3b68aa4ee None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:24.244794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f646a-ad2a-4698-8364-63d3b68aa4ee None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:24.245265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f646a-ad2a-4698-8364-63d3b68aa4ee None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986/users/408be52fb987442f88fc591ee9126f2f/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:24.247537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f646a-ad2a-4698-8364-63d3b68aa4ee None admin] RBAC: Authorizing `identity:create_grant(project_id=a91190fa89bd41f49048e6128973a986, user_id=408be52fb987442f88fc591ee9126f2f, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:24.249301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f646a-ad2a-4698-8364-63d3b68aa4ee None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:24.274304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ac2ef-98a2-4aa4-952c-97dcfea7eb53 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:24.313643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3169766-1392-4fc1-859a-2ad4367022c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:24.314681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3169766-1392-4fc1-859a-2ad4367022c4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:24.314966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3169766-1392-4fc1-859a-2ad4367022c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:24.315306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3169766-1392-4fc1-859a-2ad4367022c4 None admin] PATH_INFO: `/v3/users/408be52fb987442f88fc591ee9126f2f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:24.316362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 118/231] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:37:24 2022] PUT /identity/v3/projects/a91190fa89bd41f49048e6128973a986/users/408be52fb987442f88fc591ee9126f2f/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:24.322235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3169766-1392-4fc1-859a-2ad4367022c4 None admin] RBAC: Authorizing `identity:get_user(user_id=408be52fb987442f88fc591ee9126f2f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:24.323850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3169766-1392-4fc1-859a-2ad4367022c4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:24.325982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 114/232] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:24 2022] GET /identity/v3/users/408be52fb987442f88fc591ee9126f2f => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:24.340078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f646a-ad2a-4698-8364-63d3b68aa4ee None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:24.439465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:24.475616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f646a-ad2a-4698-8364-63d3b68aa4ee None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:24.552024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa446e-211c-4851-91c1-50438ec341ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:24.552888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa446e-211c-4851-91c1-50438ec341ad None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:24.553170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa446e-211c-4851-91c1-50438ec341ad None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:24.553448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa446e-211c-4851-91c1-50438ec341ad None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:24.554665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa446e-211c-4851-91c1-50438ec341ad None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:24.556689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa446e-211c-4851-91c1-50438ec341ad None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:24.558603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 119/233] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:24 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:24.564294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3169766-1392-4fc1-859a-2ad4367022c4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:24.669801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3169766-1392-4fc1-859a-2ad4367022c4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:24.683220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:24.705211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e737d-5f43-46d0-91da-e834e81821f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:24.706169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e737d-5f43-46d0-91da-e834e81821f3 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:24.706454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e737d-5f43-46d0-91da-e834e81821f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:24.706732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e737d-5f43-46d0-91da-e834e81821f3 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986/users/408be52fb987442f88fc591ee9126f2f/roles/7583ad887d8b4052bd11783371c338b4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:24.708950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e737d-5f43-46d0-91da-e834e81821f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a91190fa89bd41f49048e6128973a986, user_id=408be52fb987442f88fc591ee9126f2f, role_id=7583ad887d8b4052bd11783371c338b4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:24.718693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e737d-5f43-46d0-91da-e834e81821f3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:24.821684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 115/234] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:37:24 2022] PUT /identity/v3/projects/a91190fa89bd41f49048e6128973a986/users/408be52fb987442f88fc591ee9126f2f/roles/7583ad887d8b4052bd11783371c338b4 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:25.489476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6df177-fc35-43e4-af1b-0e972d26de95 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:25.490012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6df177-fc35-43e4-af1b-0e972d26de95 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:25.490087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6df177-fc35-43e4-af1b-0e972d26de95 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:25.491657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 120/235] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:25.496766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95162d41-8ab6-4b11-a61c-5716a11f0bb8 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:25.497064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95162d41-8ab6-4b11-a61c-5716a11f0bb8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:25.497347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95162d41-8ab6-4b11-a61c-5716a11f0bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:25.582912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-95162d41-8ab6-4b11-a61c-5716a11f0bb8 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:25.584110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-95162d41-8ab6-4b11-a61c-5716a11f0bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:25.672379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 116/236] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:25.702846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc853e-dce2-4648-b1cd-0c31f4c7169b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:25.703469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc853e-dce2-4648-b1cd-0c31f4c7169b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:25.703942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc853e-dce2-4648-b1cd-0c31f4c7169b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:25.705504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 121/237] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:25.710174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc853e-dce2-4648-b1cd-0c31f4c7169b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:25.721813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c514a-60ae-424f-83c0-8029a6f8cdaa None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:25.722136 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c514a-60ae-424f-83c0-8029a6f8cdaa None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:25.722420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c514a-60ae-424f-83c0-8029a6f8cdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:25.759011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1b9c514a-60ae-424f-83c0-8029a6f8cdaa None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:25.760108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1b9c514a-60ae-424f-83c0-8029a6f8cdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:25.837093 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc853e-dce2-4648-b1cd-0c31f4c7169b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:25.861640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 117/238] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:25.886626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c514a-60ae-424f-83c0-8029a6f8cdaa None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:25.902215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29b1c2-9cb9-49f2-b3b6-d9d48d7badf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:25.903033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29b1c2-9cb9-49f2-b3b6-d9d48d7badf0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:25.903317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29b1c2-9cb9-49f2-b3b6-d9d48d7badf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:25.903594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29b1c2-9cb9-49f2-b3b6-d9d48d7badf0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:25.928641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29b1c2-9cb9-49f2-b3b6-d9d48d7badf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:25.941990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7f29b1c2-9cb9-49f2-b3b6-d9d48d7badf0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:25.947605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:25.948465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:25.949244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:25.952087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 122/239] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:25.957115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29b1c2-9cb9-49f2-b3b6-d9d48d7badf0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:25.973224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:26.074468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29b1c2-9cb9-49f2-b3b6-d9d48d7badf0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:26.113905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c514a-60ae-424f-83c0-8029a6f8cdaa None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:26.116131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995db56-048b-4e29-a7c9-2747726c6780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:26.116810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995db56-048b-4e29-a7c9-2747726c6780 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:26.117095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995db56-048b-4e29-a7c9-2747726c6780 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:26.117368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995db56-048b-4e29-a7c9-2747726c6780 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:26.118803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995db56-048b-4e29-a7c9-2747726c6780 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:26.119111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995db56-048b-4e29-a7c9-2747726c6780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:26.120519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995db56-048b-4e29-a7c9-2747726c6780 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:26.139155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 123/240] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:26.142132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995db56-048b-4e29-a7c9-2747726c6780 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:26.172598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bfa51-d30f-4d5c-84e9-be0500dfbb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:26.173401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bfa51-d30f-4d5c-84e9-be0500dfbb98 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:26.173686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bfa51-d30f-4d5c-84e9-be0500dfbb98 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:26.173987 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bfa51-d30f-4d5c-84e9-be0500dfbb98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:26.175155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bfa51-d30f-4d5c-84e9-be0500dfbb98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:26.176601 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bfa51-d30f-4d5c-84e9-be0500dfbb98 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:26.188576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 118/241] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:26.189047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bfa51-d30f-4d5c-84e9-be0500dfbb98 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:26.266537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:26.328173 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995db56-048b-4e29-a7c9-2747726c6780 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:26.367121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bfa51-d30f-4d5c-84e9-be0500dfbb98 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:26.367551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:26.391788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdbe35-6e1f-4458-a290-2b1d3ae143ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:26.392676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdbe35-6e1f-4458-a290-2b1d3ae143ab None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:26.392955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdbe35-6e1f-4458-a290-2b1d3ae143ab None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:26.393225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdbe35-6e1f-4458-a290-2b1d3ae143ab None admin] PATH_INFO: `/v3/users/f2f99a2e383049bbbf0d63ba5a9235fb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:26.416768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec9a09-19ee-4c8b-8d22-7ca632cdd15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:26.417440 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec9a09-19ee-4c8b-8d22-7ca632cdd15f None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:26.417716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec9a09-19ee-4c8b-8d22-7ca632cdd15f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:26.418011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec9a09-19ee-4c8b-8d22-7ca632cdd15f None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:26.418721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec9a09-19ee-4c8b-8d22-7ca632cdd15f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:26.420174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec9a09-19ee-4c8b-8d22-7ca632cdd15f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:26.421622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdbe35-6e1f-4458-a290-2b1d3ae143ab None admin] RBAC: Authorizing `identity:get_user(user_id=f2f99a2e383049bbbf0d63ba5a9235fb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:26.423105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdbe35-6e1f-4458-a290-2b1d3ae143ab None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:26.424836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 124/242] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:26 2022] GET /identity/v3/users/f2f99a2e383049bbbf0d63ba5a9235fb => generated 297 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:26.431616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdbe35-6e1f-4458-a290-2b1d3ae143ab None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:26.514032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 119/243] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:26 2022] POST /identity/v3/users => generated 293 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:26.553600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdbe35-6e1f-4458-a290-2b1d3ae143ab None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:26.582333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:26.590486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43aa3e-9e27-4a0a-b22d-8f68d44507c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:26.591143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43aa3e-9e27-4a0a-b22d-8f68d44507c5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:26.591423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43aa3e-9e27-4a0a-b22d-8f68d44507c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:26.591695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43aa3e-9e27-4a0a-b22d-8f68d44507c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:26.663079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43aa3e-9e27-4a0a-b22d-8f68d44507c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:26.664608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43aa3e-9e27-4a0a-b22d-8f68d44507c5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:26.770166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 125/244] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:26.779124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec9a09-19ee-4c8b-8d22-7ca632cdd15f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:26.908137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec9a09-19ee-4c8b-8d22-7ca632cdd15f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:26.942046 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc1533-a995-4210-b020-ebce077efbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:26.942909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc1533-a995-4210-b020-ebce077efbcd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:26.943189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc1533-a995-4210-b020-ebce077efbcd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:26.943463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc1533-a995-4210-b020-ebce077efbcd None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:26.944647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc1533-a995-4210-b020-ebce077efbcd None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:26.946617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc1533-a995-4210-b020-ebce077efbcd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:26.948311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 120/245] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:26 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:26.954090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43aa3e-9e27-4a0a-b22d-8f68d44507c5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:26.980188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:27.037184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c9820-5040-4349-a123-f42edc864100 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.037502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c9820-5040-4349-a123-f42edc864100 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.037736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c9820-5040-4349-a123-f42edc864100 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.039382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 121/246] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:27.045052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdce1a6-1414-482d-8c09-37b59d742a93 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.045335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdce1a6-1414-482d-8c09-37b59d742a93 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.045610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdce1a6-1414-482d-8c09-37b59d742a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.066186 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4fdce1a6-1414-482d-8c09-37b59d742a93 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:27.066501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43aa3e-9e27-4a0a-b22d-8f68d44507c5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:27.067361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4fdce1a6-1414-482d-8c09-37b59d742a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:27.096936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:27.114114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c743439-68f1-45f4-8dce-d827e8843da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:27.115143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c743439-68f1-45f4-8dce-d827e8843da7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.115494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c743439-68f1-45f4-8dce-d827e8843da7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.115838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c743439-68f1-45f4-8dce-d827e8843da7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.116746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c743439-68f1-45f4-8dce-d827e8843da7 None admin] RBAC: Adding query filter params (role.id=3394eca9fdfe4986becd456e050c26e3, scope.project.id=a91190fa89bd41f49048e6128973a986, user.id=f2f99a2e383049bbbf0d63ba5a9235fb) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:27.117126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c743439-68f1-45f4-8dce-d827e8843da7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:27.119344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c743439-68f1-45f4-8dce-d827e8843da7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:27.125368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 126/247] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:26 2022] GET /identity/v3/role_assignments?user.id=f2f99a2e383049bbbf0d63ba5a9235fb&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=3394eca9fdfe4986becd456e050c26e3 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:27.159243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 122/248] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:27.187273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c743439-68f1-45f4-8dce-d827e8843da7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:27.197333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218152cb-9b66-418b-9512-a9d49546fb36 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.197520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218152cb-9b66-418b-9512-a9d49546fb36 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.197839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218152cb-9b66-418b-9512-a9d49546fb36 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.199570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 123/249] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:27.205060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1d911-8936-43aa-b3f0-a14440af4c9c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.205344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1d911-8936-43aa-b3f0-a14440af4c9c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.205625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1d911-8936-43aa-b3f0-a14440af4c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.231382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-19a1d911-8936-43aa-b3f0-a14440af4c9c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:27.232619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-19a1d911-8936-43aa-b3f0-a14440af4c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:27.320021 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c743439-68f1-45f4-8dce-d827e8843da7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:27.322730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:27.340566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 124/250] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:27.366221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dd4aa-681c-41b2-9a5a-8a32f7b9babf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:27.367108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dd4aa-681c-41b2-9a5a-8a32f7b9babf None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.367389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dd4aa-681c-41b2-9a5a-8a32f7b9babf None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.367693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dd4aa-681c-41b2-9a5a-8a32f7b9babf None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.368631 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1d911-8936-43aa-b3f0-a14440af4c9c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:27.379330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9dd4aa-681c-41b2-9a5a-8a32f7b9babf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0f9dd4aa-681c-41b2-9a5a-8a32f7b9babf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:27.387335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:27.388204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:27.388963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 127/251] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:27.394539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dd4aa-681c-41b2-9a5a-8a32f7b9babf None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:27.466921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:27.485354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1d911-8936-43aa-b3f0-a14440af4c9c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:27.500638 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dd4aa-681c-41b2-9a5a-8a32f7b9babf None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:27.550599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28b10c-2dd5-44d1-bc01-9efa6eb3e3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:27.551268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28b10c-2dd5-44d1-bc01-9efa6eb3e3f6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.551599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28b10c-2dd5-44d1-bc01-9efa6eb3e3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.551877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28b10c-2dd5-44d1-bc01-9efa6eb3e3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.552613 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28b10c-2dd5-44d1-bc01-9efa6eb3e3f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:27.553067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28b10c-2dd5-44d1-bc01-9efa6eb3e3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:27.554614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28b10c-2dd5-44d1-bc01-9efa6eb3e3f6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:27.564226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 128/252] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:27.570451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1647d3c-5c61-43b1-8942-cb7f00e48cff None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.571553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1647d3c-5c61-43b1-8942-cb7f00e48cff None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.571971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1647d3c-5c61-43b1-8942-cb7f00e48cff None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.572879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f450b-52d1-4c9f-b92d-260af1ccfaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:27.573792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f450b-52d1-4c9f-b92d-260af1ccfaca None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.574128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 129/253] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:27.574266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f450b-52d1-4c9f-b92d-260af1ccfaca None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.574648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f450b-52d1-4c9f-b92d-260af1ccfaca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.575372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1647d3c-5c61-43b1-8942-cb7f00e48cff None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:27.590264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:27.600995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f450b-52d1-4c9f-b92d-260af1ccfaca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b19f450b-52d1-4c9f-b92d-260af1ccfaca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:27.620843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:27.622087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:27.622898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 125/254] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:37:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:27.625745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac11f2-7112-4be2-be8d-3c1936e9d26d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.625974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac11f2-7112-4be2-be8d-3c1936e9d26d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.626207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac11f2-7112-4be2-be8d-3c1936e9d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.652075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-acac11f2-7112-4be2-be8d-3c1936e9d26d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:27.653265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-acac11f2-7112-4be2-be8d-3c1936e9d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:27.756357 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1647d3c-5c61-43b1-8942-cb7f00e48cff None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:27.759534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 126/255] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:27.760386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac11f2-7112-4be2-be8d-3c1936e9d26d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:27.828924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af37606-05c9-46c6-a266-a677a7d96cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:27.829817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af37606-05c9-46c6-a266-a677a7d96cd1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.830129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af37606-05c9-46c6-a266-a677a7d96cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.830407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af37606-05c9-46c6-a266-a677a7d96cd1 None admin] PATH_INFO: `/v3/users/5d98ea05ab1746508ce20c43554bb5e0` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.833095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af37606-05c9-46c6-a266-a677a7d96cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=5d98ea05ab1746508ce20c43554bb5e0)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:27.834604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af37606-05c9-46c6-a266-a677a7d96cd1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:27.840121 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 130/256] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:27 2022] GET /identity/v3/users/5d98ea05ab1746508ce20c43554bb5e0 => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:27.841098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af37606-05c9-46c6-a266-a677a7d96cd1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:27.866308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:27.893697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac11f2-7112-4be2-be8d-3c1936e9d26d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:27.937232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8459a8-6e3b-48ad-b676-da7a851a0ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:27.938297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8459a8-6e3b-48ad-b676-da7a851a0ea6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:27.940129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8459a8-6e3b-48ad-b676-da7a851a0ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:27.940615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8459a8-6e3b-48ad-b676-da7a851a0ea6 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:27.941594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8459a8-6e3b-48ad-b676-da7a851a0ea6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:27.942904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8459a8-6e3b-48ad-b676-da7a851a0ea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:27.945247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8459a8-6e3b-48ad-b676-da7a851a0ea6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:27.962542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 127/257] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:27.964308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8459a8-6e3b-48ad-b676-da7a851a0ea6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:27.998564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:28.044841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af37606-05c9-46c6-a266-a677a7d96cd1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:28.107583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8459a8-6e3b-48ad-b676-da7a851a0ea6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:28.109039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68295c-184c-4b8c-9984-7ef7fa76b82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:28.110104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68295c-184c-4b8c-9984-7ef7fa76b82e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:28.112125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68295c-184c-4b8c-9984-7ef7fa76b82e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:28.112624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68295c-184c-4b8c-9984-7ef7fa76b82e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:28.135078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68295c-184c-4b8c-9984-7ef7fa76b82e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0f68295c-184c-4b8c-9984-7ef7fa76b82e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:28.145280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:28.146206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:28.146978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:28.150009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 131/258] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:28.151076 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68295c-184c-4b8c-9984-7ef7fa76b82e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:28.192411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd34eb-d4fc-4b03-9cb5-24a75322ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:28.194512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd34eb-d4fc-4b03-9cb5-24a75322ba5f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:28.198513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd34eb-d4fc-4b03-9cb5-24a75322ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:28.199029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd34eb-d4fc-4b03-9cb5-24a75322ba5f None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:28.200601 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd34eb-d4fc-4b03-9cb5-24a75322ba5f None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:28.202279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd34eb-d4fc-4b03-9cb5-24a75322ba5f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:28.205811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 128/259] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:27 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:28.208893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd34eb-d4fc-4b03-9cb5-24a75322ba5f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:28.217089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:28.322594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68295c-184c-4b8c-9984-7ef7fa76b82e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:28.364380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51529de4-cee2-46f9-b612-921b3a2de13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:28.365258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529de4-cee2-46f9-b612-921b3a2de13d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:28.365541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529de4-cee2-46f9-b612-921b3a2de13d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:28.365816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529de4-cee2-46f9-b612-921b3a2de13d None admin] PATH_INFO: `/v3/users/408be52fb987442f88fc591ee9126f2f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:28.368379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51529de4-cee2-46f9-b612-921b3a2de13d None admin] RBAC: Authorizing `identity:get_user(user_id=408be52fb987442f88fc591ee9126f2f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:28.369843 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51529de4-cee2-46f9-b612-921b3a2de13d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:28.371494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 132/260] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:28 2022] GET /identity/v3/users/408be52fb987442f88fc591ee9126f2f => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:28.372231 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51529de4-cee2-46f9-b612-921b3a2de13d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:28.396032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:28.438059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd34eb-d4fc-4b03-9cb5-24a75322ba5f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:28.504712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51529de4-cee2-46f9-b612-921b3a2de13d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:28.521799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236bbb4-ec72-44eb-8576-eb293d4cee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:28.522902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236bbb4-ec72-44eb-8576-eb293d4cee16 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:28.523393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236bbb4-ec72-44eb-8576-eb293d4cee16 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:28.523961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236bbb4-ec72-44eb-8576-eb293d4cee16 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:28.524940 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236bbb4-ec72-44eb-8576-eb293d4cee16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:28.525463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236bbb4-ec72-44eb-8576-eb293d4cee16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:28.527186 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236bbb4-ec72-44eb-8576-eb293d4cee16 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:28.541648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 129/261] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:28.542579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236bbb4-ec72-44eb-8576-eb293d4cee16 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:28.553838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadedfa-b324-4664-bccf-655e2edc99db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:28.554764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadedfa-b324-4664-bccf-655e2edc99db None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:28.555829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadedfa-b324-4664-bccf-655e2edc99db None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:28.557504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadedfa-b324-4664-bccf-655e2edc99db None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986/users/5d98ea05ab1746508ce20c43554bb5e0/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:28.562760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadedfa-b324-4664-bccf-655e2edc99db None admin] RBAC: Authorizing `identity:create_grant(project_id=a91190fa89bd41f49048e6128973a986, user_id=5d98ea05ab1746508ce20c43554bb5e0, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:28.564388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadedfa-b324-4664-bccf-655e2edc99db None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:28.626557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:28.646689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 133/262] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:37:28 2022] PUT /identity/v3/projects/a91190fa89bd41f49048e6128973a986/users/5d98ea05ab1746508ce20c43554bb5e0/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:28.647413 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadedfa-b324-4664-bccf-655e2edc99db None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:28.703262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236bbb4-ec72-44eb-8576-eb293d4cee16 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:28.782956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f6cca-ce47-4e52-92d3-5cc19c51c6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:28.783617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f6cca-ce47-4e52-92d3-5cc19c51c6b2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:28.783899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f6cca-ce47-4e52-92d3-5cc19c51c6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:28.784172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f6cca-ce47-4e52-92d3-5cc19c51c6b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:28.792659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:28.838671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadedfa-b324-4664-bccf-655e2edc99db None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:28.881834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33e38e-9fc7-4a04-8ba5-de746a124f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:28.882680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33e38e-9fc7-4a04-8ba5-de746a124f88 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:28.882965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33e38e-9fc7-4a04-8ba5-de746a124f88 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:28.883310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33e38e-9fc7-4a04-8ba5-de746a124f88 None admin] PATH_INFO: `/v3/users/408be52fb987442f88fc591ee9126f2f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:28.884509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33e38e-9fc7-4a04-8ba5-de746a124f88 None admin] RBAC: Authorizing `identity:get_user(user_id=408be52fb987442f88fc591ee9126f2f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:28.885998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd33e38e-9fc7-4a04-8ba5-de746a124f88 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:28.888267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 134/263] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:28 2022] GET /identity/v3/users/408be52fb987442f88fc591ee9126f2f => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:28.894086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33e38e-9fc7-4a04-8ba5-de746a124f88 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:28.996331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f6cca-ce47-4e52-92d3-5cc19c51c6b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:28.997795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f6cca-ce47-4e52-92d3-5cc19c51c6b2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:29.028816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33e38e-9fc7-4a04-8ba5-de746a124f88 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:29.074806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824acdb-de00-4819-ade3-f7c811dc5647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:29.077089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824acdb-de00-4819-ade3-f7c811dc5647 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:29.078581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824acdb-de00-4819-ade3-f7c811dc5647 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:29.079071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824acdb-de00-4819-ade3-f7c811dc5647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:29.085816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:29.122253 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:29.149475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 130/264] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:29.154566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f6cca-ce47-4e52-92d3-5cc19c51c6b2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:29.212513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824acdb-de00-4819-ade3-f7c811dc5647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:29.214465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824acdb-de00-4819-ade3-f7c811dc5647 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:29.264463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f6cca-ce47-4e52-92d3-5cc19c51c6b2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:29.310596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105d86a-0217-4b9b-a1a9-69528899bde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:29.311481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105d86a-0217-4b9b-a1a9-69528899bde7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:29.311776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105d86a-0217-4b9b-a1a9-69528899bde7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:29.312057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105d86a-0217-4b9b-a1a9-69528899bde7 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:29.313308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105d86a-0217-4b9b-a1a9-69528899bde7 None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:29.314832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105d86a-0217-4b9b-a1a9-69528899bde7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:29.316871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 131/265] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:29 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:29.322278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105d86a-0217-4b9b-a1a9-69528899bde7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:29.361759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:29.402002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 135/266] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:29.407627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824acdb-de00-4819-ade3-f7c811dc5647 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:29.438686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105d86a-0217-4b9b-a1a9-69528899bde7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:29.476792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:29.507407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07aea0c-3740-484e-b7e4-5dae09c92df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:29.508493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07aea0c-3740-484e-b7e4-5dae09c92df7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:29.508972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07aea0c-3740-484e-b7e4-5dae09c92df7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:29.509443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07aea0c-3740-484e-b7e4-5dae09c92df7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:29.510611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07aea0c-3740-484e-b7e4-5dae09c92df7 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=a91190fa89bd41f49048e6128973a986, user.id=408be52fb987442f88fc591ee9126f2f) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:29.511140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07aea0c-3740-484e-b7e4-5dae09c92df7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:29.512808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07aea0c-3740-484e-b7e4-5dae09c92df7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:29.526789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 132/267] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:29 2022] GET /identity/v3/role_assignments?user.id=408be52fb987442f88fc591ee9126f2f&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=fe4d636f5c4842f18569347911e6cdec => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:29.580401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824acdb-de00-4819-ade3-f7c811dc5647 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:29.630884 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f04f68-6d9d-4ed8-9823-c610dbac629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:29.632122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f04f68-6d9d-4ed8-9823-c610dbac629d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:29.632605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f04f68-6d9d-4ed8-9823-c610dbac629d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:29.633078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f04f68-6d9d-4ed8-9823-c610dbac629d None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:29.638887 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f04f68-6d9d-4ed8-9823-c610dbac629d None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:29.640774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f04f68-6d9d-4ed8-9823-c610dbac629d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:29.643264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 136/268] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:29 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:29.649992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07aea0c-3740-484e-b7e4-5dae09c92df7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:29.796854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:29.801558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07aea0c-3740-484e-b7e4-5dae09c92df7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:29.844499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7d9b1-b7c3-4aeb-a714-0add4e066e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:29.845577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7d9b1-b7c3-4aeb-a714-0add4e066e01 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:29.846138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7d9b1-b7c3-4aeb-a714-0add4e066e01 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:29.846615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7d9b1-b7c3-4aeb-a714-0add4e066e01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:29.847670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7d9b1-b7c3-4aeb-a714-0add4e066e01 None admin] RBAC: Adding query filter params (role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=a91190fa89bd41f49048e6128973a986, user.id=408be52fb987442f88fc591ee9126f2f) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:29.848241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7d9b1-b7c3-4aeb-a714-0add4e066e01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:29.849873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7d9b1-b7c3-4aeb-a714-0add4e066e01 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:29.861254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 133/269] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:29 2022] GET /identity/v3/role_assignments?user.id=408be52fb987442f88fc591ee9126f2f&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=7583ad887d8b4052bd11783371c338b4 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:30.094568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f1bd2-1be6-42b2-a466-f691caf0521c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.095266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f1bd2-1be6-42b2-a466-f691caf0521c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.095752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f1bd2-1be6-42b2-a466-f691caf0521c None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.097606 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 137/270] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:30.102701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c817ee8-4bd6-4cc7-a329-5cb0acdb14e7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.103557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c817ee8-4bd6-4cc7-a329-5cb0acdb14e7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.104055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c817ee8-4bd6-4cc7-a329-5cb0acdb14e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.131050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2c817ee8-4bd6-4cc7-a329-5cb0acdb14e7 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:30.132594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2c817ee8-4bd6-4cc7-a329-5cb0acdb14e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:30.167102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f2f9-4f50-4cff-a04e-f16df4c3b295 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.167404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f2f9-4f50-4cff-a04e-f16df4c3b295 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.167674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320f2f9-4f50-4cff-a04e-f16df4c3b295 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.169372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 138/271] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:30.174356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ccb24-8c13-4018-858f-48378785f3cf None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.174646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ccb24-8c13-4018-858f-48378785f3cf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.174929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ccb24-8c13-4018-858f-48378785f3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.206240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cb2ccb24-8c13-4018-858f-48378785f3cf None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:30.207442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cb2ccb24-8c13-4018-858f-48378785f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:30.223438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 134/272] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:30.258895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c817ee8-4bd6-4cc7-a329-5cb0acdb14e7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:30.319341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 139/273] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:30.342087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ccb24-8c13-4018-858f-48378785f3cf None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:30.387323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:30.396875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:30.407267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c817ee8-4bd6-4cc7-a329-5cb0acdb14e7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:30.450455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7187f49-cd87-41c2-b8ce-c89eaf0b14dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:30.451255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7187f49-cd87-41c2-b8ce-c89eaf0b14dd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.451529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7187f49-cd87-41c2-b8ce-c89eaf0b14dd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.451801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7187f49-cd87-41c2-b8ce-c89eaf0b14dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.462932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7187f49-cd87-41c2-b8ce-c89eaf0b14dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a7187f49-cd87-41c2-b8ce-c89eaf0b14dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:30.474344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:30.481153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:30.481947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:30.482616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ccb24-8c13-4018-858f-48378785f3cf None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:30.482688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 135/274] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:30.482688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7634-f3b4-4325-b2d9-848592d27561 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.482688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7634-f3b4-4325-b2d9-848592d27561 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.482688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7634-f3b4-4325-b2d9-848592d27561 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.484536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 136/275] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:30.485575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7634-f3b4-4325-b2d9-848592d27561 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:30.515387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc11e67-d681-4c3b-90d5-b52f1557b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:30.516186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc11e67-d681-4c3b-90d5-b52f1557b6eb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.516473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc11e67-d681-4c3b-90d5-b52f1557b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.516752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc11e67-d681-4c3b-90d5-b52f1557b6eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.527841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc11e67-d681-4c3b-90d5-b52f1557b6eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8fc11e67-d681-4c3b-90d5-b52f1557b6eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:30.535823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:30.536684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:30.537440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 140/276] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:30.543155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fc3c3-b9be-44aa-b15f-e8b0aec34cd2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.543491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fc3c3-b9be-44aa-b15f-e8b0aec34cd2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.543779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fc3c3-b9be-44aa-b15f-e8b0aec34cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.559862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e82fc3c3-b9be-44aa-b15f-e8b0aec34cd2 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:30.561041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e82fc3c3-b9be-44aa-b15f-e8b0aec34cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:30.595669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7634-f3b4-4325-b2d9-848592d27561 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:30.637984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 141/277] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:30.637984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fc3c3-b9be-44aa-b15f-e8b0aec34cd2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:30.639347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f6582-0dd2-455a-8331-27eb9cbb8dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:30.640148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f6582-0dd2-455a-8331-27eb9cbb8dbb None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.640442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f6582-0dd2-455a-8331-27eb9cbb8dbb None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.640724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f6582-0dd2-455a-8331-27eb9cbb8dbb None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.641480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f6582-0dd2-455a-8331-27eb9cbb8dbb None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:30.641793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f6582-0dd2-455a-8331-27eb9cbb8dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:30.643285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f6582-0dd2-455a-8331-27eb9cbb8dbb None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:30.653925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 137/278] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:30.658816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f6582-0dd2-455a-8331-27eb9cbb8dbb None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:30.666804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:30.676094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:30.745033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fc3c3-b9be-44aa-b15f-e8b0aec34cd2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:30.768217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f6582-0dd2-455a-8331-27eb9cbb8dbb None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:30.779449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24166fab-2886-4eda-b07f-355ab455379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:30.780120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24166fab-2886-4eda-b07f-355ab455379a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.780404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24166fab-2886-4eda-b07f-355ab455379a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.780681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24166fab-2886-4eda-b07f-355ab455379a None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.781429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24166fab-2886-4eda-b07f-355ab455379a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:30.781736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24166fab-2886-4eda-b07f-355ab455379a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:30.783162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24166fab-2886-4eda-b07f-355ab455379a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:30.791839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 142/279] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:30.792469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24166fab-2886-4eda-b07f-355ab455379a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:30.808468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb5614-64f1-40cf-a05a-288f817aac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:30.809347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb5614-64f1-40cf-a05a-288f817aac6d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.809633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb5614-64f1-40cf-a05a-288f817aac6d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.809933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb5614-64f1-40cf-a05a-288f817aac6d None admin] PATH_INFO: `/v3/users/0f567972041c496398cf32db5d0ea9a5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.823825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb5614-64f1-40cf-a05a-288f817aac6d None admin] RBAC: Authorizing `identity:get_user(user_id=0f567972041c496398cf32db5d0ea9a5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:30.825278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb5614-64f1-40cf-a05a-288f817aac6d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:30.831589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 138/280] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:30 2022] GET /identity/v3/users/0f567972041c496398cf32db5d0ea9a5 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:30.831589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb5614-64f1-40cf-a05a-288f817aac6d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:30.882472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:30.900351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:30.900510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24166fab-2886-4eda-b07f-355ab455379a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:30.929722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb5614-64f1-40cf-a05a-288f817aac6d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:30.936188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48c57e-6aaf-4577-a2a1-8019a7fb5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:30.936984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48c57e-6aaf-4577-a2a1-8019a7fb5639 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.937272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48c57e-6aaf-4577-a2a1-8019a7fb5639 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.937557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48c57e-6aaf-4577-a2a1-8019a7fb5639 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.938762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48c57e-6aaf-4577-a2a1-8019a7fb5639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:30.940312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48c57e-6aaf-4577-a2a1-8019a7fb5639 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:30.941951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 143/281] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:30.941951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48c57e-6aaf-4577-a2a1-8019a7fb5639 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:30.971247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3727e33-698f-4adf-8a4d-bfc1cd8fa65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:30.972285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3727e33-698f-4adf-8a4d-bfc1cd8fa65f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:30.972646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3727e33-698f-4adf-8a4d-bfc1cd8fa65f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:30.972932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3727e33-698f-4adf-8a4d-bfc1cd8fa65f None admin] PATH_INFO: `/v3/users/f2f99a2e383049bbbf0d63ba5a9235fb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:30.974149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3727e33-698f-4adf-8a4d-bfc1cd8fa65f None admin] RBAC: Authorizing `identity:get_user(user_id=f2f99a2e383049bbbf0d63ba5a9235fb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:30.975595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3727e33-698f-4adf-8a4d-bfc1cd8fa65f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:30.981518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 139/282] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:30 2022] GET /identity/v3/users/f2f99a2e383049bbbf0d63ba5a9235fb => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:30.981518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3727e33-698f-4adf-8a4d-bfc1cd8fa65f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:31.050901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48c57e-6aaf-4577-a2a1-8019a7fb5639 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:31.085841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0bb71-b512-4a74-a1c6-bab708bcd4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:31.087057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0bb71-b512-4a74-a1c6-bab708bcd4a5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:31.087345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0bb71-b512-4a74-a1c6-bab708bcd4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:31.087628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0bb71-b512-4a74-a1c6-bab708bcd4a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:31.088459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0bb71-b512-4a74-a1c6-bab708bcd4a5 None admin] RBAC: Adding query filter params (role.id=7bcc1da1e79942c0a21920d35347bf06, user.id=0f567972041c496398cf32db5d0ea9a5, scope.system=all) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:31.088775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0bb71-b512-4a74-a1c6-bab708bcd4a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:31.088922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3727e33-698f-4adf-8a4d-bfc1cd8fa65f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:31.090348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0bb71-b512-4a74-a1c6-bab708bcd4a5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:31.095854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 144/283] 10.222.0.42 () {56 vars in 1308 bytes} [Fri Jun 3 09:37:30 2022] GET /identity/v3/role_assignments?user.id=0f567972041c496398cf32db5d0ea9a5&scope.system=all&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:31.096635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0bb71-b512-4a74-a1c6-bab708bcd4a5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:31.124771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:31.147413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ace9f-d41f-4dba-b040-10412580ee43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:31.150202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ace9f-d41f-4dba-b040-10412580ee43 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:31.150202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ace9f-d41f-4dba-b040-10412580ee43 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:31.150202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ace9f-d41f-4dba-b040-10412580ee43 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:31.151159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ace9f-d41f-4dba-b040-10412580ee43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:31.153570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ace9f-d41f-4dba-b040-10412580ee43 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:31.167808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:31.226174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0bb71-b512-4a74-a1c6-bab708bcd4a5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:31.263659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2105bd-9dad-4781-aa7c-843b1b98d50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:31.264534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2105bd-9dad-4781-aa7c-843b1b98d50d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:31.264854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2105bd-9dad-4781-aa7c-843b1b98d50d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:31.265132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2105bd-9dad-4781-aa7c-843b1b98d50d None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:31.266395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2105bd-9dad-4781-aa7c-843b1b98d50d None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:31.267881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2105bd-9dad-4781-aa7c-843b1b98d50d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:31.269916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 145/284] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:31 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:31.275736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2105bd-9dad-4781-aa7c-843b1b98d50d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:31.293217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 140/285] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:30 2022] POST /identity/v3/users => generated 293 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:31.382323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:31.400357 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2105bd-9dad-4781-aa7c-843b1b98d50d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:31.438293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7006e-808c-4973-9fa3-1dccd9f9615b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:31.439139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7006e-808c-4973-9fa3-1dccd9f9615b None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:31.439423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7006e-808c-4973-9fa3-1dccd9f9615b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:31.439700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7006e-808c-4973-9fa3-1dccd9f9615b None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986/users/f2f99a2e383049bbbf0d63ba5a9235fb/roles/3394eca9fdfe4986becd456e050c26e3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:31.441824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7006e-808c-4973-9fa3-1dccd9f9615b None admin] RBAC: Authorizing `identity:create_grant(project_id=a91190fa89bd41f49048e6128973a986, user_id=f2f99a2e383049bbbf0d63ba5a9235fb, role_id=3394eca9fdfe4986becd456e050c26e3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:31.443658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7006e-808c-4973-9fa3-1dccd9f9615b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:31.465800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf66a3-a4c5-4153-ba20-17efcd229e43 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:31.466421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf66a3-a4c5-4153-ba20-17efcd229e43 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:31.466706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf66a3-a4c5-4153-ba20-17efcd229e43 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:31.468324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 141/286] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:31.474150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc9d0f-557d-49dd-9bb5-149c54f75b32 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:31.474443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc9d0f-557d-49dd-9bb5-149c54f75b32 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:31.474725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc9d0f-557d-49dd-9bb5-149c54f75b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:31.493274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ecfc9d0f-557d-49dd-9bb5-149c54f75b32 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:31.494496 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ecfc9d0f-557d-49dd-9bb5-149c54f75b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:31.566708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 146/287] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:37:31 2022] PUT /identity/v3/projects/a91190fa89bd41f49048e6128973a986/users/f2f99a2e383049bbbf0d63ba5a9235fb/roles/3394eca9fdfe4986becd456e050c26e3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:37:31.567110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 142/288] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:31.591641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7006e-808c-4973-9fa3-1dccd9f9615b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:31.721264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7006e-808c-4973-9fa3-1dccd9f9615b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:31.760610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9a04c-8ab7-4b11-831b-b66dca20df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:31.761414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9a04c-8ab7-4b11-831b-b66dca20df41 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:31.761696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9a04c-8ab7-4b11-831b-b66dca20df41 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:31.762012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9a04c-8ab7-4b11-831b-b66dca20df41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:31.774408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9a04c-8ab7-4b11-831b-b66dca20df41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-67b9a04c-8ab7-4b11-831b-b66dca20df41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:31.782353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:31.783365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:31.784598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 147/289] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:31.790041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc9d0f-557d-49dd-9bb5-149c54f75b32 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:31.894570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc9d0f-557d-49dd-9bb5-149c54f75b32 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:31.923804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:31.932009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c298956-5f48-4cea-8c02-9d58b60a02d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:31.932679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c298956-5f48-4cea-8c02-9d58b60a02d2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:31.932967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c298956-5f48-4cea-8c02-9d58b60a02d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:31.933242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c298956-5f48-4cea-8c02-9d58b60a02d2 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:31.934027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c298956-5f48-4cea-8c02-9d58b60a02d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:31.934337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c298956-5f48-4cea-8c02-9d58b60a02d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:31.935718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c298956-5f48-4cea-8c02-9d58b60a02d2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:31.944545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 143/290] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:31.950607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9a04c-8ab7-4b11-831b-b66dca20df41 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:31.955251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:32.070099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9a04c-8ab7-4b11-831b-b66dca20df41 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:32.106822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855461b-94c4-4c10-856f-818577518516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:32.107701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855461b-94c4-4c10-856f-818577518516 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:32.108029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855461b-94c4-4c10-856f-818577518516 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:32.108315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855461b-94c4-4c10-856f-818577518516 None admin] PATH_INFO: `/v3/users/5d98ea05ab1746508ce20c43554bb5e0` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:32.109546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855461b-94c4-4c10-856f-818577518516 None admin] RBAC: Authorizing `identity:get_user(user_id=5d98ea05ab1746508ce20c43554bb5e0)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:32.111054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855461b-94c4-4c10-856f-818577518516 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:32.112686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 148/291] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:31 2022] GET /identity/v3/users/5d98ea05ab1746508ce20c43554bb5e0 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:32.118367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c298956-5f48-4cea-8c02-9d58b60a02d2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:32.238606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c298956-5f48-4cea-8c02-9d58b60a02d2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:32.280829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff160e5-c6e6-4413-afff-4f642c6f6552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:32.281823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff160e5-c6e6-4413-afff-4f642c6f6552 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:32.282328 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff160e5-c6e6-4413-afff-4f642c6f6552 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:32.282841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff160e5-c6e6-4413-afff-4f642c6f6552 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:32.340781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:32.352906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff160e5-c6e6-4413-afff-4f642c6f6552 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:32.354483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff160e5-c6e6-4413-afff-4f642c6f6552 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:32.481841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 144/292] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:32.488875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855461b-94c4-4c10-856f-818577518516 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:32.603526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:32.642222 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855461b-94c4-4c10-856f-818577518516 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:32.688782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b841db0-3051-4e21-8510-f89a8005de28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:32.689647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b841db0-3051-4e21-8510-f89a8005de28 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:32.690029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b841db0-3051-4e21-8510-f89a8005de28 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:32.690312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b841db0-3051-4e21-8510-f89a8005de28 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:32.694591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b841db0-3051-4e21-8510-f89a8005de28 None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:32.696084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b841db0-3051-4e21-8510-f89a8005de28 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:32.697774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 149/293] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:32 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:32.703483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff160e5-c6e6-4413-afff-4f642c6f6552 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:32.816567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:32.828174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff160e5-c6e6-4413-afff-4f642c6f6552 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:32.870597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b451435-cf60-4962-ab82-e481b2dd964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:32.871187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b451435-cf60-4962-ab82-e481b2dd964d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:32.871472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b451435-cf60-4962-ab82-e481b2dd964d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:32.871750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b451435-cf60-4962-ab82-e481b2dd964d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:32.872578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b451435-cf60-4962-ab82-e481b2dd964d None admin] RBAC: Adding query filter params (role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=a91190fa89bd41f49048e6128973a986, user.id=5d98ea05ab1746508ce20c43554bb5e0) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:32.873070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b451435-cf60-4962-ab82-e481b2dd964d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:32.874555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b451435-cf60-4962-ab82-e481b2dd964d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:32.880687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 145/294] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:32 2022] GET /identity/v3/role_assignments?user.id=5d98ea05ab1746508ce20c43554bb5e0&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:33.675466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e735ac-332b-4036-ae3f-e9a7c339b1a4 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:33.675976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e735ac-332b-4036-ae3f-e9a7c339b1a4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:33.676051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e735ac-332b-4036-ae3f-e9a7c339b1a4 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:33.677655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 150/295] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:33.683516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0304200-b90f-4572-a023-1717e7152465 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:33.683950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0304200-b90f-4572-a023-1717e7152465 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:33.684125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0304200-b90f-4572-a023-1717e7152465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:33.701772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a0304200-b90f-4572-a023-1717e7152465 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:33.703007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a0304200-b90f-4572-a023-1717e7152465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:33.778826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 146/296] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:33.797549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e735ac-332b-4036-ae3f-e9a7c339b1a4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:33.909643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e735ac-332b-4036-ae3f-e9a7c339b1a4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:33.952260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59b42b-e832-4b5c-83d6-59911d50a451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:33.952989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59b42b-e832-4b5c-83d6-59911d50a451 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:33.953275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59b42b-e832-4b5c-83d6-59911d50a451 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:33.953553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59b42b-e832-4b5c-83d6-59911d50a451 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:33.964595 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59b42b-e832-4b5c-83d6-59911d50a451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:33.971287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-9d59b42b-e832-4b5c-83d6-59911d50a451 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:33.973043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:33.973918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:33.974699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 151/297] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:33.980321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0304200-b90f-4572-a023-1717e7152465 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:34.069929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:34.088899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0304200-b90f-4572-a023-1717e7152465 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:34.124150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb0d54-d13f-418a-83c3-386eede32bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:34.124818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0d54-d13f-418a-83c3-386eede32bca None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.125099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0d54-d13f-418a-83c3-386eede32bca None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.125378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0d54-d13f-418a-83c3-386eede32bca None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.126152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb0d54-d13f-418a-83c3-386eede32bca None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:34.126457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb0d54-d13f-418a-83c3-386eede32bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:34.127831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb0d54-d13f-418a-83c3-386eede32bca None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:34.141004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 147/298] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:34.142393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59b42b-e832-4b5c-83d6-59911d50a451 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:34.155263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573fb42-e262-4745-bde5-67d62518a58c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.155546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573fb42-e262-4745-bde5-67d62518a58c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.155828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573fb42-e262-4745-bde5-67d62518a58c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.157574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 148/299] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:34.163117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7fe25-898a-4b1e-b594-a7629ef1f846 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.163404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7fe25-898a-4b1e-b594-a7629ef1f846 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.163685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7fe25-898a-4b1e-b594-a7629ef1f846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.180917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-10d7fe25-898a-4b1e-b594-a7629ef1f846 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:34.182130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-10d7fe25-898a-4b1e-b594-a7629ef1f846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:34.252007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 149/300] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:34.262489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59b42b-e832-4b5c-83d6-59911d50a451 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:34.274862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7fe25-898a-4b1e-b594-a7629ef1f846 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:34.307649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd455e-187e-4ed5-9772-bba905fdb51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:34.308542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd455e-187e-4ed5-9772-bba905fdb51f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.308795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd455e-187e-4ed5-9772-bba905fdb51f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.309069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd455e-187e-4ed5-9772-bba905fdb51f None admin] PATH_INFO: `/v3/users/0f567972041c496398cf32db5d0ea9a5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.310278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd455e-187e-4ed5-9772-bba905fdb51f None admin] RBAC: Authorizing `identity:get_user(user_id=0f567972041c496398cf32db5d0ea9a5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:34.311720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd455e-187e-4ed5-9772-bba905fdb51f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:34.313485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 152/301] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:34 2022] GET /identity/v3/users/0f567972041c496398cf32db5d0ea9a5 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:34.314569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a19d-be29-49e1-b0ea-d362b681c82d None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.314852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a19d-be29-49e1-b0ea-d362b681c82d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.315127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a19d-be29-49e1-b0ea-d362b681c82d None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.316460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 153/302] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:34.319634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34a19d-be29-49e1-b0ea-d362b681c82d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:34.362978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:34.386907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7fe25-898a-4b1e-b594-a7629ef1f846 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:34.389114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:34.422642 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b050bdb-51d6-4003-aaa7-e2ba671aa43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:34.423320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34a19d-be29-49e1-b0ea-d362b681c82d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:34.423436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b050bdb-51d6-4003-aaa7-e2ba671aa43f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.423696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b050bdb-51d6-4003-aaa7-e2ba671aa43f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.423975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b050bdb-51d6-4003-aaa7-e2ba671aa43f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.425289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b050bdb-51d6-4003-aaa7-e2ba671aa43f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:34.427339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b050bdb-51d6-4003-aaa7-e2ba671aa43f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:34.428982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 150/303] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:34.430209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378a283-5fb2-419c-afe7-e640c60badbf None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.430493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378a283-5fb2-419c-afe7-e640c60badbf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.430772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378a283-5fb2-419c-afe7-e640c60badbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.447822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5378a283-5fb2-419c-afe7-e640c60badbf None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:34.449014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5378a283-5fb2-419c-afe7-e640c60badbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:34.460908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966eac8-5e33-4d9f-a0d0-3c6aebba41e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:34.461818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966eac8-5e33-4d9f-a0d0-3c6aebba41e2 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.462132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966eac8-5e33-4d9f-a0d0-3c6aebba41e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.462411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966eac8-5e33-4d9f-a0d0-3c6aebba41e2 None admin] PATH_INFO: `/v3/system/users/0f567972041c496398cf32db5d0ea9a5/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.463861 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966eac8-5e33-4d9f-a0d0-3c6aebba41e2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0f567972041c496398cf32db5d0ea9a5, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:34.465226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966eac8-5e33-4d9f-a0d0-3c6aebba41e2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:34.534796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 151/304] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:34.535711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 154/305] 10.222.0.42 () {58 vars in 1334 bytes} [Fri Jun 3 09:37:34 2022] PUT /identity/v3/system/users/0f567972041c496398cf32db5d0ea9a5/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:37:34.536096 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e281a-c3c4-4e56-be66-91898992592e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.536362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966eac8-5e33-4d9f-a0d0-3c6aebba41e2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:34.536462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e281a-c3c4-4e56-be66-91898992592e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.536648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e281a-c3c4-4e56-be66-91898992592e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.537949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 152/306] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:34.544669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199be06-b181-443e-98cb-c30b04aadadd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.544958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199be06-b181-443e-98cb-c30b04aadadd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.545234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199be06-b181-443e-98cb-c30b04aadadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.567685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e199be06-b181-443e-98cb-c30b04aadadd None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:34.568935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e199be06-b181-443e-98cb-c30b04aadadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:34.648406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 153/307] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:34.649644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199be06-b181-443e-98cb-c30b04aadadd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:34.652285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:34.687137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966eac8-5e33-4d9f-a0d0-3c6aebba41e2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:34.719448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:34.724866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0586e1f-22ff-4137-8655-656026182476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:34.725542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0586e1f-22ff-4137-8655-656026182476 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.725818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0586e1f-22ff-4137-8655-656026182476 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.726155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0586e1f-22ff-4137-8655-656026182476 None admin] PATH_INFO: `/v3/groups` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.727385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0586e1f-22ff-4137-8655-656026182476 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:34.728953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0586e1f-22ff-4137-8655-656026182476 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:34.767765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199be06-b181-443e-98cb-c30b04aadadd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:34.802149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b6119-7e58-4031-856c-23377d1763b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:34.802958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b6119-7e58-4031-856c-23377d1763b9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.803232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b6119-7e58-4031-856c-23377d1763b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.803503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b6119-7e58-4031-856c-23377d1763b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.813363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b6119-7e58-4031-856c-23377d1763b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:34.820984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 155/308] 10.222.0.42 () {60 vars in 1134 bytes} [Fri Jun 3 09:37:34 2022] POST /identity/v3/groups => generated 229 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-927b6119-7e58-4031-856c-23377d1763b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:34.821148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:34.824953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:34.825748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:34.826416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0586e1f-22ff-4137-8655-656026182476 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:34.827188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 154/309] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:34.828153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b6119-7e58-4031-856c-23377d1763b9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:34.900696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:34.945333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b6119-7e58-4031-856c-23377d1763b9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:34.952339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0586e1f-22ff-4137-8655-656026182476 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:34.983729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847e7e-9714-434e-b572-025907703d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:34.984771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847e7e-9714-434e-b572-025907703d3c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.985201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847e7e-9714-434e-b572-025907703d3c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.985672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847e7e-9714-434e-b572-025907703d3c None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.986638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847e7e-9714-434e-b572-025907703d3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:34.987094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847e7e-9714-434e-b572-025907703d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:34.988662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847e7e-9714-434e-b572-025907703d3c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:34.990139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90049a26-3822-495e-a1fa-bf1be0e243be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:34.991048 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90049a26-3822-495e-a1fa-bf1be0e243be None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:34.991331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90049a26-3822-495e-a1fa-bf1be0e243be None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:34.991615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90049a26-3822-495e-a1fa-bf1be0e243be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:34.999749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 155/310] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:35.004828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847e7e-9714-434e-b572-025907703d3c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:35.006597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90049a26-3822-495e-a1fa-bf1be0e243be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-90049a26-3822-495e-a1fa-bf1be0e243be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:35.016333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:35.017304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:35.018338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 156/311] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:35.023323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90049a26-3822-495e-a1fa-bf1be0e243be None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:35.090455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:35.140449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847e7e-9714-434e-b572-025907703d3c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:35.145270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90049a26-3822-495e-a1fa-bf1be0e243be None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:35.163957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:35.176761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fca71-3444-4459-91bf-10e7ea8b5062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:35.178744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fca71-3444-4459-91bf-10e7ea8b5062 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:35.178744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fca71-3444-4459-91bf-10e7ea8b5062 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:35.178744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fca71-3444-4459-91bf-10e7ea8b5062 None admin] PATH_INFO: `/v3/users/f2f99a2e383049bbbf0d63ba5a9235fb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:35.179791 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fca71-3444-4459-91bf-10e7ea8b5062 None admin] RBAC: Authorizing `identity:get_user(user_id=f2f99a2e383049bbbf0d63ba5a9235fb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:35.181795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fca71-3444-4459-91bf-10e7ea8b5062 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:35.183726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 156/312] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:35 2022] GET /identity/v3/users/f2f99a2e383049bbbf0d63ba5a9235fb => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:35.183933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d19ec-e70b-45ac-a848-711e09e81a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:35.184760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d19ec-e70b-45ac-a848-711e09e81a32 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:35.185187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d19ec-e70b-45ac-a848-711e09e81a32 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:35.185597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d19ec-e70b-45ac-a848-711e09e81a32 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:35.186536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d19ec-e70b-45ac-a848-711e09e81a32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:35.186977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d19ec-e70b-45ac-a848-711e09e81a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:35.188704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d19ec-e70b-45ac-a848-711e09e81a32 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:35.189087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fca71-3444-4459-91bf-10e7ea8b5062 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:35.197603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 157/313] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:35.204052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d19ec-e70b-45ac-a848-711e09e81a32 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:35.311616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fca71-3444-4459-91bf-10e7ea8b5062 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:35.317115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d19ec-e70b-45ac-a848-711e09e81a32 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:35.331152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:35.349572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7308f-6e1b-4615-b195-6971b2545b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:35.350595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7308f-6e1b-4615-b195-6971b2545b2c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:35.351061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7308f-6e1b-4615-b195-6971b2545b2c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:35.351539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7308f-6e1b-4615-b195-6971b2545b2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:35.352626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6a689-6c74-4c2b-bfe4-e5957716f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:35.353560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6a689-6c74-4c2b-bfe4-e5957716f50e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:35.353917 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6a689-6c74-4c2b-bfe4-e5957716f50e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:35.354270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6a689-6c74-4c2b-bfe4-e5957716f50e None admin] PATH_INFO: `/v3/users/b526a66f917c4368a3cd585db35d29e2` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:35.368515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6a689-6c74-4c2b-bfe4-e5957716f50e None admin] RBAC: Authorizing `identity:get_user(user_id=b526a66f917c4368a3cd585db35d29e2)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:35.370152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6a689-6c74-4c2b-bfe4-e5957716f50e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:35.371958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 158/314] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:35 2022] GET /identity/v3/users/b526a66f917c4368a3cd585db35d29e2 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:35.377571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6a689-6c74-4c2b-bfe4-e5957716f50e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:35.426263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:35.429063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7308f-6e1b-4615-b195-6971b2545b2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:35.430948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7308f-6e1b-4615-b195-6971b2545b2c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:35.482364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6a689-6c74-4c2b-bfe4-e5957716f50e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:35.518262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fef5-40f6-411e-82e3-d0ce5c38c8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:35.519291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fef5-40f6-411e-82e3-d0ce5c38c8e0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:35.519805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fef5-40f6-411e-82e3-d0ce5c38c8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:35.520286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7fef5-40f6-411e-82e3-d0ce5c38c8e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:35.521289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7fef5-40f6-411e-82e3-d0ce5c38c8e0 None admin] RBAC: Adding query filter params (role.id=3394eca9fdfe4986becd456e050c26e3, user.id=b526a66f917c4368a3cd585db35d29e2, scope.system=all) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:35.521778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7fef5-40f6-411e-82e3-d0ce5c38c8e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:35.524163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7fef5-40f6-411e-82e3-d0ce5c38c8e0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:35.530411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 159/315] 10.222.0.42 () {56 vars in 1308 bytes} [Fri Jun 3 09:37:35 2022] GET /identity/v3/role_assignments?user.id=b526a66f917c4368a3cd585db35d29e2&scope.system=all&role.id=3394eca9fdfe4986becd456e050c26e3 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:35.541786 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 157/316] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:35.546988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fef5-40f6-411e-82e3-d0ce5c38c8e0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:35.602836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:35.658874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7fef5-40f6-411e-82e3-d0ce5c38c8e0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:35.699431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04b3bf-78e0-4650-84d3-50d95ae44bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:35.700314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04b3bf-78e0-4650-84d3-50d95ae44bf4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:35.700599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04b3bf-78e0-4650-84d3-50d95ae44bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:35.700878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04b3bf-78e0-4650-84d3-50d95ae44bf4 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:35.702160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04b3bf-78e0-4650-84d3-50d95ae44bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:35.703651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04b3bf-78e0-4650-84d3-50d95ae44bf4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:35.705736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 160/317] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:35 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:35.711913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7308f-6e1b-4615-b195-6971b2545b2c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:35.811478 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:35.820789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7308f-6e1b-4615-b195-6971b2545b2c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:35.855615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad07fdb-d1c0-4c8d-af2c-5d993dd1d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:35.857127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad07fdb-d1c0-4c8d-af2c-5d993dd1d502 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:35.857988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad07fdb-d1c0-4c8d-af2c-5d993dd1d502 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:35.858275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad07fdb-d1c0-4c8d-af2c-5d993dd1d502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:35.859116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad07fdb-d1c0-4c8d-af2c-5d993dd1d502 None admin] RBAC: Adding query filter params (role.id=3394eca9fdfe4986becd456e050c26e3, scope.project.id=a91190fa89bd41f49048e6128973a986, user.id=f2f99a2e383049bbbf0d63ba5a9235fb) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:35.859430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad07fdb-d1c0-4c8d-af2c-5d993dd1d502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:35.860869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad07fdb-d1c0-4c8d-af2c-5d993dd1d502 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:35.867166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 158/318] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:37:35 2022] GET /identity/v3/role_assignments?user.id=f2f99a2e383049bbbf0d63ba5a9235fb&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=3394eca9fdfe4986becd456e050c26e3 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:36.822079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb75ef-b7ee-4655-9a25-6ac4f341f9f3 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:36.822582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb75ef-b7ee-4655-9a25-6ac4f341f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:36.822687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb75ef-b7ee-4655-9a25-6ac4f341f9f3 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:36.824039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 161/319] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:36.829307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eee42-f3c0-4bc7-a6f7-f94ddadb8b22 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:36.829679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eee42-f3c0-4bc7-a6f7-f94ddadb8b22 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:36.829838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eee42-f3c0-4bc7-a6f7-f94ddadb8b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:36.847457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6c1eee42-f3c0-4bc7-a6f7-f94ddadb8b22 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:36.848654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6c1eee42-f3c0-4bc7-a6f7-f94ddadb8b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:36.920265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 159/320] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:36.943758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb75ef-b7ee-4655-9a25-6ac4f341f9f3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:37.045503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb75ef-b7ee-4655-9a25-6ac4f341f9f3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:37.047866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:37.078922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd097e-a1de-48e0-bf4c-24de60090022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:37.079719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd097e-a1de-48e0-bf4c-24de60090022 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.080004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd097e-a1de-48e0-bf4c-24de60090022 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.080282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd097e-a1de-48e0-bf4c-24de60090022 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.091531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd097e-a1de-48e0-bf4c-24de60090022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-14cd097e-a1de-48e0-bf4c-24de60090022 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:37.099341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:37.100182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:37.100956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 162/321] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:37.107684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eee42-f3c0-4bc7-a6f7-f94ddadb8b22 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:37.192251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:37.207735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eee42-f3c0-4bc7-a6f7-f94ddadb8b22 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:37.225071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa337edf-3fba-4bf3-88e9-e63baf571dd3 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.225359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa337edf-3fba-4bf3-88e9-e63baf571dd3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.225636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa337edf-3fba-4bf3-88e9-e63baf571dd3 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.227732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 163/322] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:37.232436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605877a-911a-49b0-8812-70c6748a6791 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.232721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605877a-911a-49b0-8812-70c6748a6791 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.232999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605877a-911a-49b0-8812-70c6748a6791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.245527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5482b-ba2c-4e24-8b08-c269e791d127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:37.246480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5482b-ba2c-4e24-8b08-c269e791d127 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.246965 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5482b-ba2c-4e24-8b08-c269e791d127 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.247415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5482b-ba2c-4e24-8b08-c269e791d127 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.248344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5482b-ba2c-4e24-8b08-c269e791d127 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:37.248828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5482b-ba2c-4e24-8b08-c269e791d127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:37.249754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6605877a-911a-49b0-8812-70c6748a6791 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:37.250608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5482b-ba2c-4e24-8b08-c269e791d127 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:37.250955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6605877a-911a-49b0-8812-70c6748a6791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:37.259697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 160/323] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:37.265258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5482b-ba2c-4e24-8b08-c269e791d127 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:37.320178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 164/324] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:37.342217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605877a-911a-49b0-8812-70c6748a6791 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:37.368071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5482b-ba2c-4e24-8b08-c269e791d127 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:37.403991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cb753-7b1a-4046-8a2c-ac08e950f168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:37.405097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cb753-7b1a-4046-8a2c-ac08e950f168 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.405524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cb753-7b1a-4046-8a2c-ac08e950f168 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.406017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cb753-7b1a-4046-8a2c-ac08e950f168 None admin] PATH_INFO: `/v3/users/0f567972041c496398cf32db5d0ea9a5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.407361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cb753-7b1a-4046-8a2c-ac08e950f168 None admin] RBAC: Authorizing `identity:get_user(user_id=0f567972041c496398cf32db5d0ea9a5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:37.408946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cb753-7b1a-4046-8a2c-ac08e950f168 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:37.410783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 161/325] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:37 2022] GET /identity/v3/users/0f567972041c496398cf32db5d0ea9a5 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:37.417469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cb753-7b1a-4046-8a2c-ac08e950f168 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:37.439640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:37.439879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:37.447242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605877a-911a-49b0-8812-70c6748a6791 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:37.482084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb113d83-2225-4cd3-aaf4-a439242e7482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:37.482874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb113d83-2225-4cd3-aaf4-a439242e7482 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.483161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb113d83-2225-4cd3-aaf4-a439242e7482 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.483436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb113d83-2225-4cd3-aaf4-a439242e7482 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.484603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb113d83-2225-4cd3-aaf4-a439242e7482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:37.486064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb113d83-2225-4cd3-aaf4-a439242e7482 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:37.487710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 165/326] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:37.493082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb113d83-2225-4cd3-aaf4-a439242e7482 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:37.528159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cb753-7b1a-4046-8a2c-ac08e950f168 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:37.567856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446e457-1c3e-4566-aed3-e9cc27d40f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:37.568813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446e457-1c3e-4566-aed3-e9cc27d40f45 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.569276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446e457-1c3e-4566-aed3-e9cc27d40f45 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.569851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446e457-1c3e-4566-aed3-e9cc27d40f45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.570997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446e457-1c3e-4566-aed3-e9cc27d40f45 None admin] RBAC: Adding query filter params (role.id=7bcc1da1e79942c0a21920d35347bf06, user.id=0f567972041c496398cf32db5d0ea9a5, scope.system=all) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:37.571257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446e457-1c3e-4566-aed3-e9cc27d40f45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:37.572878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446e457-1c3e-4566-aed3-e9cc27d40f45 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:37.585578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 162/327] 10.222.0.42 () {56 vars in 1308 bytes} [Fri Jun 3 09:37:37 2022] GET /identity/v3/role_assignments?user.id=0f567972041c496398cf32db5d0ea9a5&scope.system=all&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 510 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:37.599634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb113d83-2225-4cd3-aaf4-a439242e7482 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:37.638702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e6b49-f5ef-4e08-95d0-7a5e2e52b197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:37.639380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e6b49-f5ef-4e08-95d0-7a5e2e52b197 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.639662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e6b49-f5ef-4e08-95d0-7a5e2e52b197 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.639935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e6b49-f5ef-4e08-95d0-7a5e2e52b197 None admin] PATH_INFO: `/v3/groups` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.640622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e6b49-f5ef-4e08-95d0-7a5e2e52b197 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:37.642128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e6b49-f5ef-4e08-95d0-7a5e2e52b197 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:37.736072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 166/328] 10.222.0.42 () {60 vars in 1134 bytes} [Fri Jun 3 09:37:37 2022] POST /identity/v3/groups => generated 226 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:37.883347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86c5bc-cd7c-4ca9-b040-831a9e433ac7 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.883873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86c5bc-cd7c-4ca9-b040-831a9e433ac7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.883873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86c5bc-cd7c-4ca9-b040-831a9e433ac7 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.885255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 163/329] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:37.890922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c8c6-f7a6-4422-b43d-d59bd6466b36 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:37.891387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c8c6-f7a6-4422-b43d-d59bd6466b36 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:37.891490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c8c6-f7a6-4422-b43d-d59bd6466b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:37.909300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5db6c8c6-f7a6-4422-b43d-d59bd6466b36 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:37.910563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5db6c8c6-f7a6-4422-b43d-d59bd6466b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:37.928848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:37.980195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 167/330] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:38.002615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86c5bc-cd7c-4ca9-b040-831a9e433ac7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:38.088390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:38.115557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86c5bc-cd7c-4ca9-b040-831a9e433ac7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:38.151274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fbdca-1868-45dd-b2a5-e81bd38ee64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:38.152016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fbdca-1868-45dd-b2a5-e81bd38ee64c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:38.152301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fbdca-1868-45dd-b2a5-e81bd38ee64c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:38.152578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fbdca-1868-45dd-b2a5-e81bd38ee64c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:38.163452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062fbdca-1868-45dd-b2a5-e81bd38ee64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-062fbdca-1868-45dd-b2a5-e81bd38ee64c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:38.172145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:38.173204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:38.174329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 164/331] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:38.179677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6c8c6-f7a6-4422-b43d-d59bd6466b36 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:38.283030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6c8c6-f7a6-4422-b43d-d59bd6466b36 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:38.320962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390276ba-f15b-4815-8538-fbcdc9df041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:38.321895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390276ba-f15b-4815-8538-fbcdc9df041e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:38.322326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390276ba-f15b-4815-8538-fbcdc9df041e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:38.322746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390276ba-f15b-4815-8538-fbcdc9df041e None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:38.323663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390276ba-f15b-4815-8538-fbcdc9df041e None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:38.324115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390276ba-f15b-4815-8538-fbcdc9df041e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:38.325657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390276ba-f15b-4815-8538-fbcdc9df041e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:38.334943 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 168/332] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:38.341783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fbdca-1868-45dd-b2a5-e81bd38ee64c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:38.468424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fbdca-1868-45dd-b2a5-e81bd38ee64c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:38.497773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28032f0-79f8-4777-b793-698fb0a06828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:38.498742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28032f0-79f8-4777-b793-698fb0a06828 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:38.499051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28032f0-79f8-4777-b793-698fb0a06828 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:38.499335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28032f0-79f8-4777-b793-698fb0a06828 None admin] PATH_INFO: `/v3/users/b526a66f917c4368a3cd585db35d29e2` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:38.500630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28032f0-79f8-4777-b793-698fb0a06828 None admin] RBAC: Authorizing `identity:get_user(user_id=b526a66f917c4368a3cd585db35d29e2)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:38.502119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28032f0-79f8-4777-b793-698fb0a06828 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:38.504108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 165/333] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:38 2022] GET /identity/v3/users/b526a66f917c4368a3cd585db35d29e2 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:38.509330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390276ba-f15b-4815-8538-fbcdc9df041e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:38.532131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:38.628475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390276ba-f15b-4815-8538-fbcdc9df041e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:38.668946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbdb2a-3d37-48d4-a292-ea79ffa81f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:38.669825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbdb2a-3d37-48d4-a292-ea79ffa81f30 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:38.670141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbdb2a-3d37-48d4-a292-ea79ffa81f30 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:38.670425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbdb2a-3d37-48d4-a292-ea79ffa81f30 None admin] PATH_INFO: `/v3/system/users/b526a66f917c4368a3cd585db35d29e2/roles/3394eca9fdfe4986becd456e050c26e3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:38.672129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbdb2a-3d37-48d4-a292-ea79ffa81f30 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b526a66f917c4368a3cd585db35d29e2, role_id=3394eca9fdfe4986becd456e050c26e3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:38.673788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbdb2a-3d37-48d4-a292-ea79ffa81f30 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:38.826643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 169/334] 10.222.0.42 () {58 vars in 1334 bytes} [Fri Jun 3 09:37:38 2022] PUT /identity/v3/system/users/b526a66f917c4368a3cd585db35d29e2/roles/3394eca9fdfe4986becd456e050c26e3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:40.439483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4508300-abf8-4454-910b-6a42550bec40 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:40.440001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4508300-abf8-4454-910b-6a42550bec40 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:40.440086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4508300-abf8-4454-910b-6a42550bec40 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:40.441801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 166/335] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:40.447378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983afd8-ff39-4e8c-bc9d-743fef74be18 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:40.451068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983afd8-ff39-4e8c-bc9d-743fef74be18 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:40.451068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983afd8-ff39-4e8c-bc9d-743fef74be18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:40.470868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9983afd8-ff39-4e8c-bc9d-743fef74be18 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:40.472080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9983afd8-ff39-4e8c-bc9d-743fef74be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:40.556754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 170/336] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:40.583100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4508300-abf8-4454-910b-6a42550bec40 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:40.586606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:40.704607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4508300-abf8-4454-910b-6a42550bec40 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:40.751477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ecd67-301b-4396-a61e-cb019bf83ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:40.752307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ecd67-301b-4396-a61e-cb019bf83ea2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:40.752593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ecd67-301b-4396-a61e-cb019bf83ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:40.752876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ecd67-301b-4396-a61e-cb019bf83ea2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:40.764208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ecd67-301b-4396-a61e-cb019bf83ea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-576ecd67-301b-4396-a61e-cb019bf83ea2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:40.776784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:40.777707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:40.778545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 167/337] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:40.785513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983afd8-ff39-4e8c-bc9d-743fef74be18 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:40.796618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:40.905281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983afd8-ff39-4e8c-bc9d-743fef74be18 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:40.953556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcdc28-ca77-40fa-a028-ef5f7c6ea5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:40.954263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcdc28-ca77-40fa-a028-ef5f7c6ea5c9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:40.954553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcdc28-ca77-40fa-a028-ef5f7c6ea5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:40.954883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcdc28-ca77-40fa-a028-ef5f7c6ea5c9 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:40.955587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcdc28-ca77-40fa-a028-ef5f7c6ea5c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:40.955892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcdc28-ca77-40fa-a028-ef5f7c6ea5c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:40.957273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcdc28-ca77-40fa-a028-ef5f7c6ea5c9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:40.969741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 171/338] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:40.975303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ecd67-301b-4396-a61e-cb019bf83ea2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:41.054466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:41.106233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ecd67-301b-4396-a61e-cb019bf83ea2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:41.148648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e559b4e-6636-4e44-903a-471ec3fe3421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:41.149310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e559b4e-6636-4e44-903a-471ec3fe3421 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:41.149599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e559b4e-6636-4e44-903a-471ec3fe3421 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:41.149899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e559b4e-6636-4e44-903a-471ec3fe3421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:41.233243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e559b4e-6636-4e44-903a-471ec3fe3421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:41.234678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e559b4e-6636-4e44-903a-471ec3fe3421 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:41.306445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:41.355743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 168/339] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:41.362538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcdc28-ca77-40fa-a028-ef5f7c6ea5c9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:41.432028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a1586-6799-46c6-a1b9-60c6bf8ff5bb None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:41.432386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a1586-6799-46c6-a1b9-60c6bf8ff5bb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:41.432587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a1586-6799-46c6-a1b9-60c6bf8ff5bb None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:41.434198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 169/340] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:41.439280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26796b6a-99ae-4564-9cc9-226415144404 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:41.439571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26796b6a-99ae-4564-9cc9-226415144404 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:41.440088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26796b6a-99ae-4564-9cc9-226415144404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:41.465040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-26796b6a-99ae-4564-9cc9-226415144404 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:41.466760 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-26796b6a-99ae-4564-9cc9-226415144404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:41.470247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:41.516753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcdc28-ca77-40fa-a028-ef5f7c6ea5c9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:41.568592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 170/341] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:41.571001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be249dbf-61a3-408e-8c10-a3029162ae45 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:41.571513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be249dbf-61a3-408e-8c10-a3029162ae45 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:41.571798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be249dbf-61a3-408e-8c10-a3029162ae45 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:41.575287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 171/342] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:41.583395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b916104-79e3-40bd-ac39-67c8250c63c6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:41.583688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b916104-79e3-40bd-ac39-67c8250c63c6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:41.583970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b916104-79e3-40bd-ac39-67c8250c63c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:41.595702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50533078-6fd7-4871-9a91-e5d2ad323957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:41.596578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50533078-6fd7-4871-9a91-e5d2ad323957 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:41.596871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50533078-6fd7-4871-9a91-e5d2ad323957 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:41.598385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50533078-6fd7-4871-9a91-e5d2ad323957 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:41.600558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50533078-6fd7-4871-9a91-e5d2ad323957 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:41.602194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50533078-6fd7-4871-9a91-e5d2ad323957 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:41.604242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 172/343] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:41 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:41.605089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50533078-6fd7-4871-9a91-e5d2ad323957 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:41.620506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1b916104-79e3-40bd-ac39-67c8250c63c6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:41.621706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1b916104-79e3-40bd-ac39-67c8250c63c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:41.762639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 172/344] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:41.763518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b916104-79e3-40bd-ac39-67c8250c63c6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:41.764247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50533078-6fd7-4871-9a91-e5d2ad323957 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:41.808200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:41.850978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d7118-61d8-4bd7-a416-ecc9b09771a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:41.852714 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d7118-61d8-4bd7-a416-ecc9b09771a1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:41.853112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d7118-61d8-4bd7-a416-ecc9b09771a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:41.853491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d7118-61d8-4bd7-a416-ecc9b09771a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:41.879627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:41.894505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d7118-61d8-4bd7-a416-ecc9b09771a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:41.900939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b916104-79e3-40bd-ac39-67c8250c63c6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0b4d7118-61d8-4bd7-a416-ecc9b09771a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:41.902358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:41.903207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:41.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 173/345] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:41.905079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d7118-61d8-4bd7-a416-ecc9b09771a1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:41.937851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253da2c9-e159-44e7-9846-62756275e7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:41.938757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253da2c9-e159-44e7-9846-62756275e7dc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:41.939048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253da2c9-e159-44e7-9846-62756275e7dc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:41.939332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253da2c9-e159-44e7-9846-62756275e7dc None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:41.945970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253da2c9-e159-44e7-9846-62756275e7dc None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:41.947684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253da2c9-e159-44e7-9846-62756275e7dc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:41.949590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 173/346] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:41 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:41.950606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253da2c9-e159-44e7-9846-62756275e7dc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:42.046431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:42.060787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d7118-61d8-4bd7-a416-ecc9b09771a1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:42.063055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253da2c9-e159-44e7-9846-62756275e7dc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:42.101369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6225b2-b26e-4d22-a40e-9dd06617219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:42.102608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6225b2-b26e-4d22-a40e-9dd06617219c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.103034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6225b2-b26e-4d22-a40e-9dd06617219c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.103448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6225b2-b26e-4d22-a40e-9dd06617219c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.105244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910810d4-5c94-4642-a81a-b725f02beaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:42.106477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910810d4-5c94-4642-a81a-b725f02beaf1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.106964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910810d4-5c94-4642-a81a-b725f02beaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.107405 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910810d4-5c94-4642-a81a-b725f02beaf1 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.108318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910810d4-5c94-4642-a81a-b725f02beaf1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:42.108772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910810d4-5c94-4642-a81a-b725f02beaf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:42.110596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910810d4-5c94-4642-a81a-b725f02beaf1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:42.114939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6225b2-b26e-4d22-a40e-9dd06617219c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:42.125931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 174/347] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:42.125931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910810d4-5c94-4642-a81a-b725f02beaf1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-1c6225b2-b26e-4d22-a40e-9dd06617219c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:42.128819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:42.129684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:42.130510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:42.132637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 174/348] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:42.134755 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96a1ce-154d-4d08-9788-e3a91f55b851 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.135365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96a1ce-154d-4d08-9788-e3a91f55b851 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.136091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96a1ce-154d-4d08-9788-e3a91f55b851 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.139761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 175/349] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:42.140677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96a1ce-154d-4d08-9788-e3a91f55b851 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:42.216641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:42.258113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910810d4-5c94-4642-a81a-b725f02beaf1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:42.276555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96a1ce-154d-4d08-9788-e3a91f55b851 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:42.306817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee489a4c-f042-46ae-8ab2-03b6430218f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:42.308112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee489a4c-f042-46ae-8ab2-03b6430218f4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.308403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee489a4c-f042-46ae-8ab2-03b6430218f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.308682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee489a4c-f042-46ae-8ab2-03b6430218f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.310011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee489a4c-f042-46ae-8ab2-03b6430218f4 None admin] RBAC: Adding query filter params (group.id=604ee7930c4b48c486e6532caf56a94e, role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=ca796ca2de804171942d2631cccd56c7) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:42.310493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee489a4c-f042-46ae-8ab2-03b6430218f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:42.312908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee489a4c-f042-46ae-8ab2-03b6430218f4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:42.320826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 175/350] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:42 2022] GET /identity/v3/role_assignments?group.id=604ee7930c4b48c486e6532caf56a94e&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=7583ad887d8b4052bd11783371c338b4 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:42.321179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2f748-bb7b-4005-b096-a1c28d26e55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:42.321646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee489a4c-f042-46ae-8ab2-03b6430218f4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:42.321938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2f748-bb7b-4005-b096-a1c28d26e55d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.322229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2f748-bb7b-4005-b096-a1c28d26e55d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.322555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2f748-bb7b-4005-b096-a1c28d26e55d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.344334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:42.411381 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2f748-bb7b-4005-b096-a1c28d26e55d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:42.413317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2f748-bb7b-4005-b096-a1c28d26e55d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:42.462056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee489a4c-f042-46ae-8ab2-03b6430218f4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:42.493061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:42.504554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6034e9-10cc-4adb-b310-4e05b785420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:42.505581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6034e9-10cc-4adb-b310-4e05b785420c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.506074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6034e9-10cc-4adb-b310-4e05b785420c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.506518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6034e9-10cc-4adb-b310-4e05b785420c None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.520421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6034e9-10cc-4adb-b310-4e05b785420c None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:42.520421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6034e9-10cc-4adb-b310-4e05b785420c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:42.520421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6034e9-10cc-4adb-b310-4e05b785420c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:42.520421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 176/351] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:42.521991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa2f31-39af-4054-ba75-70f97191417f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.522557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa2f31-39af-4054-ba75-70f97191417f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.523035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa2f31-39af-4054-ba75-70f97191417f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.542569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-8eaa2f31-39af-4054-ba75-70f97191417f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:42.544155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8eaa2f31-39af-4054-ba75-70f97191417f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:42.550067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 176/352] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:42.552581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8a5f6-fe66-47a1-8bf8-c4fd5992edfc None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.553447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8a5f6-fe66-47a1-8bf8-c4fd5992edfc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.553982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8a5f6-fe66-47a1-8bf8-c4fd5992edfc None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.557689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 177/353] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:42.558509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8a5f6-fe66-47a1-8bf8-c4fd5992edfc None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:42.630966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 177/354] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:42.631742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa2f31-39af-4054-ba75-70f97191417f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:42.638154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:42.696196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8a5f6-fe66-47a1-8bf8-c4fd5992edfc None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:42.749514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe6cf2-bffb-4d5f-afb4-8c125bfe407c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:42.750203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe6cf2-bffb-4d5f-afb4-8c125bfe407c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.750486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe6cf2-bffb-4d5f-afb4-8c125bfe407c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.750760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe6cf2-bffb-4d5f-afb4-8c125bfe407c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.765522 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa2f31-39af-4054-ba75-70f97191417f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:42.786883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:42.809211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b95be-85f9-409f-990b-1734c0f435b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:42.810156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b95be-85f9-409f-990b-1734c0f435b6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.810442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b95be-85f9-409f-990b-1734c0f435b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.810716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b95be-85f9-409f-990b-1734c0f435b6 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.813593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b95be-85f9-409f-990b-1734c0f435b6 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:42.815105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b95be-85f9-409f-990b-1734c0f435b6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:42.816975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 178/355] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:42 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:42.818033 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7925a4e-ac0f-4a57-ba24-4372abf1c277 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.818522 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7925a4e-ac0f-4a57-ba24-4372abf1c277 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.818956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7925a4e-ac0f-4a57-ba24-4372abf1c277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.836805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c7925a4e-ac0f-4a57-ba24-4372abf1c277 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:42.838221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c7925a4e-ac0f-4a57-ba24-4372abf1c277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:42.844489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe6cf2-bffb-4d5f-afb4-8c125bfe407c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:42.846224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe6cf2-bffb-4d5f-afb4-8c125bfe407c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:42.928207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 179/356] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:42.930408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2987-8cb4-405a-b8ad-56034dc1c96c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:42.930697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2987-8cb4-405a-b8ad-56034dc1c96c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:42.930976 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2987-8cb4-405a-b8ad-56034dc1c96c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:42.932235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 180/357] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:42.932925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe2987-8cb4-405a-b8ad-56034dc1c96c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:42.936533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:42.981912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 178/358] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:42.982974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe6cf2-bffb-4d5f-afb4-8c125bfe407c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.056541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe2987-8cb4-405a-b8ad-56034dc1c96c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:43.061643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:43.097238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d946933-7b33-45c7-b6fb-de7cabd687c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:43.098332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d946933-7b33-45c7-b6fb-de7cabd687c2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.100778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d946933-7b33-45c7-b6fb-de7cabd687c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.102128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d946933-7b33-45c7-b6fb-de7cabd687c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.115847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d946933-7b33-45c7-b6fb-de7cabd687c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:43.122553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe6cf2-bffb-4d5f-afb4-8c125bfe407c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-4d946933-7b33-45c7-b6fb-de7cabd687c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:43.123333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:43.124187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:43.125523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 181/359] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:43.126479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d448c1-9a72-420c-9002-df05cdb325db None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.126479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d448c1-9a72-420c-9002-df05cdb325db None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.126479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d448c1-9a72-420c-9002-df05cdb325db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.145673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d2d448c1-9a72-420c-9002-df05cdb325db None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:43.145673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d2d448c1-9a72-420c-9002-df05cdb325db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:43.173850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d29a4-c3ca-44d7-b457-56925132605a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:43.174809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d29a4-c3ca-44d7-b457-56925132605a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.177081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d29a4-c3ca-44d7-b457-56925132605a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.177615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d29a4-c3ca-44d7-b457-56925132605a None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.179141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d29a4-c3ca-44d7-b457-56925132605a None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:43.180855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d29a4-c3ca-44d7-b457-56925132605a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:43.183027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 179/360] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:42 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:43.183995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d29a4-c3ca-44d7-b457-56925132605a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.216801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:43.227265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 182/361] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:43.234456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d448c1-9a72-420c-9002-df05cdb325db None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.308161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d29a4-c3ca-44d7-b457-56925132605a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:43.338571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:43.350318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bde6f-d109-49bc-90af-1ec2916de2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:43.351840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bde6f-d109-49bc-90af-1ec2916de2df None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.352017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bde6f-d109-49bc-90af-1ec2916de2df None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.352294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bde6f-d109-49bc-90af-1ec2916de2df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.358797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d448c1-9a72-420c-9002-df05cdb325db None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:43.369489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bde6f-d109-49bc-90af-1ec2916de2df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-389bde6f-d109-49bc-90af-1ec2916de2df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:43.379580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:43.380669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:43.381575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 180/362] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:37:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:43.382414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bde6f-d109-49bc-90af-1ec2916de2df None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.402639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7937692-f226-471f-ac7a-7d52c12ad8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:43.403507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7937692-f226-471f-ac7a-7d52c12ad8ce None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.403789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7937692-f226-471f-ac7a-7d52c12ad8ce None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.404183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7937692-f226-471f-ac7a-7d52c12ad8ce None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.405464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7937692-f226-471f-ac7a-7d52c12ad8ce None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:43.407015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7937692-f226-471f-ac7a-7d52c12ad8ce None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:43.409602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 183/363] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:43 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:43.410547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7937692-f226-471f-ac7a-7d52c12ad8ce None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.488924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:43.502906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bde6f-d109-49bc-90af-1ec2916de2df None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:43.542004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7937692-f226-471f-ac7a-7d52c12ad8ce None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:43.552813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdfeee-01b1-4dc4-8e94-f8e4aa9d9b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:43.553489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdfeee-01b1-4dc4-8e94-f8e4aa9d9b89 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.553776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdfeee-01b1-4dc4-8e94-f8e4aa9d9b89 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.554095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdfeee-01b1-4dc4-8e94-f8e4aa9d9b89 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.555378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdfeee-01b1-4dc4-8e94-f8e4aa9d9b89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:43.555725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdfeee-01b1-4dc4-8e94-f8e4aa9d9b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:43.557227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdfeee-01b1-4dc4-8e94-f8e4aa9d9b89 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:43.566296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 181/364] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:43.567233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdfeee-01b1-4dc4-8e94-f8e4aa9d9b89 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.594356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671645ae-5b85-4657-baeb-a76ce8773d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:43.595070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671645ae-5b85-4657-baeb-a76ce8773d32 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.596120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671645ae-5b85-4657-baeb-a76ce8773d32 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.596120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671645ae-5b85-4657-baeb-a76ce8773d32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.596512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671645ae-5b85-4657-baeb-a76ce8773d32 None admin] RBAC: Adding query filter params (group.id=604ee7930c4b48c486e6532caf56a94e, role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=ca796ca2de804171942d2631cccd56c7) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:43.596831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671645ae-5b85-4657-baeb-a76ce8773d32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:43.599083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671645ae-5b85-4657-baeb-a76ce8773d32 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:43.605897 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 184/365] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:43 2022] GET /identity/v3/role_assignments?group.id=604ee7930c4b48c486e6532caf56a94e&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:43.610955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671645ae-5b85-4657-baeb-a76ce8773d32 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.643482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:43.712680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdfeee-01b1-4dc4-8e94-f8e4aa9d9b89 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:43.756924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69353c4d-5e3d-4b5a-9e04-24448161131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:43.757946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69353c4d-5e3d-4b5a-9e04-24448161131b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.758334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69353c4d-5e3d-4b5a-9e04-24448161131b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.758710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69353c4d-5e3d-4b5a-9e04-24448161131b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.771329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69353c4d-5e3d-4b5a-9e04-24448161131b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:43.771612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671645ae-5b85-4657-baeb-a76ce8773d32 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-69353c4d-5e3d-4b5a-9e04-24448161131b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:43.781498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:43.782700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 09:37:43.783665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:43.785571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 182/366] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:43.786577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69353c4d-5e3d-4b5a-9e04-24448161131b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.806253 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:43.825173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394243d3-4e8b-4d42-9a09-9f04d326257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:43.826127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394243d3-4e8b-4d42-9a09-9f04d326257f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.826420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394243d3-4e8b-4d42-9a09-9f04d326257f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.826701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394243d3-4e8b-4d42-9a09-9f04d326257f None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.827471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394243d3-4e8b-4d42-9a09-9f04d326257f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:43.827810 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394243d3-4e8b-4d42-9a09-9f04d326257f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:43.829196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394243d3-4e8b-4d42-9a09-9f04d326257f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:43.842333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 185/367] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:43.843219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394243d3-4e8b-4d42-9a09-9f04d326257f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.914296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69353c4d-5e3d-4b5a-9e04-24448161131b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:43.924117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:43.953788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5046961-dce8-4fcb-963b-e36729c22c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:43.955073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5046961-dce8-4fcb-963b-e36729c22c05 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:43.955786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5046961-dce8-4fcb-963b-e36729c22c05 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:43.956132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5046961-dce8-4fcb-963b-e36729c22c05 None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:43.957798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5046961-dce8-4fcb-963b-e36729c22c05 None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:43.959310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5046961-dce8-4fcb-963b-e36729c22c05 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:43.961805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 183/368] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:43 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:43.962659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5046961-dce8-4fcb-963b-e36729c22c05 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:43.982915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394243d3-4e8b-4d42-9a09-9f04d326257f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:44.029118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9d944-1e35-44a3-8447-cd3f7a3da5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:44.029956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9d944-1e35-44a3-8447-cd3f7a3da5e0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:44.030302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9d944-1e35-44a3-8447-cd3f7a3da5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:44.030596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9d944-1e35-44a3-8447-cd3f7a3da5e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:44.097135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5046961-dce8-4fcb-963b-e36729c22c05 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:44.110355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:44.127896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9d944-1e35-44a3-8447-cd3f7a3da5e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:44.130207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9d944-1e35-44a3-8447-cd3f7a3da5e0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:44.138273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8def2d-7e0c-4539-9526-2f3a5275c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:44.139041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8def2d-7e0c-4539-9526-2f3a5275c293 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:44.139602 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8def2d-7e0c-4539-9526-2f3a5275c293 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:44.139977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8def2d-7e0c-4539-9526-2f3a5275c293 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:44.140812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8def2d-7e0c-4539-9526-2f3a5275c293 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:44.141594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8def2d-7e0c-4539-9526-2f3a5275c293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:44.143027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8def2d-7e0c-4539-9526-2f3a5275c293 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:44.152970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 184/369] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:44.157129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8def2d-7e0c-4539-9526-2f3a5275c293 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:44.224874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:44.265935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 186/370] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:44.267254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9d944-1e35-44a3-8447-cd3f7a3da5e0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:44.307903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8def2d-7e0c-4539-9526-2f3a5275c293 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:44.349981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c7fa4-5e60-439c-be8d-abc1e48499d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:44.350645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c7fa4-5e60-439c-be8d-abc1e48499d8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:44.350930 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c7fa4-5e60-439c-be8d-abc1e48499d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:44.351210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c7fa4-5e60-439c-be8d-abc1e48499d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:44.403998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:44.422165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9d944-1e35-44a3-8447-cd3f7a3da5e0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:44.451193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c7fa4-5e60-439c-be8d-abc1e48499d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:44.454517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c7fa4-5e60-439c-be8d-abc1e48499d8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:44.470893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d94ae2-1a4b-4123-bb30-cdc0eba75766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:44.471837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d94ae2-1a4b-4123-bb30-cdc0eba75766 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:44.472270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d94ae2-1a4b-4123-bb30-cdc0eba75766 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:44.472694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d94ae2-1a4b-4123-bb30-cdc0eba75766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:44.473796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d94ae2-1a4b-4123-bb30-cdc0eba75766 None admin] RBAC: Adding query filter params (group.id=604ee7930c4b48c486e6532caf56a94e, role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=a91190fa89bd41f49048e6128973a986) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:44.474138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d94ae2-1a4b-4123-bb30-cdc0eba75766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:44.476204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d94ae2-1a4b-4123-bb30-cdc0eba75766 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:44.484025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 187/371] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:44 2022] GET /identity/v3/role_assignments?group.id=604ee7930c4b48c486e6532caf56a94e&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:44.484853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d94ae2-1a4b-4123-bb30-cdc0eba75766 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:44.526266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:44.643794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d94ae2-1a4b-4123-bb30-cdc0eba75766 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:44.687545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57658f-35b0-471f-8f1e-5053a82b82f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:44.688810 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57658f-35b0-471f-8f1e-5053a82b82f4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:44.689423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57658f-35b0-471f-8f1e-5053a82b82f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:44.690736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57658f-35b0-471f-8f1e-5053a82b82f4 None admin] PATH_INFO: `/v3/users/b526a66f917c4368a3cd585db35d29e2` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:44.693067 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57658f-35b0-471f-8f1e-5053a82b82f4 None admin] RBAC: Authorizing `identity:get_user(user_id=b526a66f917c4368a3cd585db35d29e2)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:44.696423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 185/372] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:44.698179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57658f-35b0-471f-8f1e-5053a82b82f4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:44.700080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 188/373] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:37:44 2022] GET /identity/v3/users/b526a66f917c4368a3cd585db35d29e2 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:44.704353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c7fa4-5e60-439c-be8d-abc1e48499d8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:44.705535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57658f-35b0-471f-8f1e-5053a82b82f4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:44.731423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:44.838517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57658f-35b0-471f-8f1e-5053a82b82f4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:44.842267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c7fa4-5e60-439c-be8d-abc1e48499d8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:44.884567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cac22-32c5-4c5e-8f23-8049d2b84a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:44.885432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cac22-32c5-4c5e-8f23-8049d2b84a0f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:44.885722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cac22-32c5-4c5e-8f23-8049d2b84a0f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:44.886031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cac22-32c5-4c5e-8f23-8049d2b84a0f None admin] PATH_INFO: `/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:44.887392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cac22-32c5-4c5e-8f23-8049d2b84a0f None admin] RBAC: Authorizing `identity:get_project(project_id=53d2a61c9fa84503ab8a6c4eb2d3f27e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:44.888831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cac22-32c5-4c5e-8f23-8049d2b84a0f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:44.889104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683be8f4-5985-49eb-8bc6-cf5558769a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:44.890020 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be8f4-5985-49eb-8bc6-cf5558769a26 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:44.890301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be8f4-5985-49eb-8bc6-cf5558769a26 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:44.890573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be8f4-5985-49eb-8bc6-cf5558769a26 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:44.891758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683be8f4-5985-49eb-8bc6-cf5558769a26 None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:44.892221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 189/374] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:44 2022] GET /identity/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:44.893009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cac22-32c5-4c5e-8f23-8049d2b84a0f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:44.893178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683be8f4-5985-49eb-8bc6-cf5558769a26 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:44.896303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 186/375] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:44 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:44.897670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed30750-824a-4aa6-9c79-82e437b6ad02 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:44.897780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed30750-824a-4aa6-9c79-82e437b6ad02 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:44.902242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed30750-824a-4aa6-9c79-82e437b6ad02 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:44.903614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 187/376] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:44.904351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed30750-824a-4aa6-9c79-82e437b6ad02 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:44.914611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:45.021388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:45.033619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed30750-824a-4aa6-9c79-82e437b6ad02 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:45.040848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cac22-32c5-4c5e-8f23-8049d2b84a0f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:45.085419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee120f54-616a-4abd-8298-27088d79dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:45.086963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee120f54-616a-4abd-8298-27088d79dee5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:45.087343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee120f54-616a-4abd-8298-27088d79dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:45.087622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee120f54-616a-4abd-8298-27088d79dee5 None admin] PATH_INFO: `/v3/groups/51186006fd854cbc846ca13ba62d9289` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:45.095596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91eddb-151e-4d7e-9fad-a620aba725e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:45.096255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91eddb-151e-4d7e-9fad-a620aba725e8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:45.096452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee120f54-616a-4abd-8298-27088d79dee5 None admin] RBAC: Authorizing `identity:get_group(group_id=51186006fd854cbc846ca13ba62d9289)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:45.096550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91eddb-151e-4d7e-9fad-a620aba725e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:45.096792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91eddb-151e-4d7e-9fad-a620aba725e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:45.097623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91eddb-151e-4d7e-9fad-a620aba725e8 None admin] RBAC: Adding query filter params (role.id=3394eca9fdfe4986becd456e050c26e3, user.id=b526a66f917c4368a3cd585db35d29e2, scope.system=all) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:45.097957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91eddb-151e-4d7e-9fad-a620aba725e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:45.098126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee120f54-616a-4abd-8298-27088d79dee5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:45.099370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91eddb-151e-4d7e-9fad-a620aba725e8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:45.099973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 188/377] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:44 2022] GET /identity/v3/groups/51186006fd854cbc846ca13ba62d9289 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:45.100666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee120f54-616a-4abd-8298-27088d79dee5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:45.112890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 190/378] 10.222.0.42 () {56 vars in 1308 bytes} [Fri Jun 3 09:37:44 2022] GET /identity/v3/role_assignments?user.id=b526a66f917c4368a3cd585db35d29e2&scope.system=all&role.id=3394eca9fdfe4986becd456e050c26e3 => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:45.114261 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70e3ac-495e-4d30-af87-2efd6db0a401 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:45.114545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70e3ac-495e-4d30-af87-2efd6db0a401 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:45.114819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70e3ac-495e-4d30-af87-2efd6db0a401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:45.134288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-8a70e3ac-495e-4d30-af87-2efd6db0a401 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:45.135537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8a70e3ac-495e-4d30-af87-2efd6db0a401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:45.195674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:45.232061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 191/379] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:45.232945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70e3ac-495e-4d30-af87-2efd6db0a401 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:45.271488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee120f54-616a-4abd-8298-27088d79dee5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:45.333503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a118ab8-f001-4772-bd8c-b30d4219e3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:45.334973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a118ab8-f001-4772-bd8c-b30d4219e3f9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:45.335269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a118ab8-f001-4772-bd8c-b30d4219e3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:45.337113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a118ab8-f001-4772-bd8c-b30d4219e3f9 None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:45.338462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a118ab8-f001-4772-bd8c-b30d4219e3f9 None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:45.340173 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a118ab8-f001-4772-bd8c-b30d4219e3f9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:45.342931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 189/380] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:45 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:45.343720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a118ab8-f001-4772-bd8c-b30d4219e3f9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:45.380876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:45.410580 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70e3ac-495e-4d30-af87-2efd6db0a401 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:45.456135 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75fd72-1735-4a47-be23-3a60c2088bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:45.457233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75fd72-1735-4a47-be23-3a60c2088bc3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:45.457514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75fd72-1735-4a47-be23-3a60c2088bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:45.457790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75fd72-1735-4a47-be23-3a60c2088bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:45.458658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75fd72-1735-4a47-be23-3a60c2088bc3 None admin] RBAC: Adding query filter params (group.id=51186006fd854cbc846ca13ba62d9289, role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=53d2a61c9fa84503ab8a6c4eb2d3f27e) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:45.458961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75fd72-1735-4a47-be23-3a60c2088bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:45.460829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75fd72-1735-4a47-be23-3a60c2088bc3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:45.468879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 192/381] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:45 2022] GET /identity/v3/role_assignments?group.id=51186006fd854cbc846ca13ba62d9289&scope.project.id=53d2a61c9fa84503ab8a6c4eb2d3f27e&role.id=fe4d636f5c4842f18569347911e6cdec => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:45.470110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75fd72-1735-4a47-be23-3a60c2088bc3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:45.477326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a118ab8-f001-4772-bd8c-b30d4219e3f9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:45.555350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd17818-e41d-4706-ad24-9d033a44954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:45.556272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd17818-e41d-4706-ad24-9d033a44954a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:45.556556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd17818-e41d-4706-ad24-9d033a44954a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:45.556834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd17818-e41d-4706-ad24-9d033a44954a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:45.573695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:45.584516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd17818-e41d-4706-ad24-9d033a44954a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:45.590051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75fd72-1735-4a47-be23-3a60c2088bc3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7bd17818-e41d-4706-ad24-9d033a44954a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:45.602193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:45.603283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:45.604047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 190/382] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:45.610442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd17818-e41d-4706-ad24-9d033a44954a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:45.634791 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a98204-ac44-4492-ae30-e25434d8f5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:45.635828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98204-ac44-4492-ae30-e25434d8f5fc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:45.636276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98204-ac44-4492-ae30-e25434d8f5fc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:45.636699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a98204-ac44-4492-ae30-e25434d8f5fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:45.637688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a98204-ac44-4492-ae30-e25434d8f5fc None admin] RBAC: Adding query filter params (group.id=604ee7930c4b48c486e6532caf56a94e, role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=a91190fa89bd41f49048e6128973a986) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:45.638298 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a98204-ac44-4492-ae30-e25434d8f5fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:45.640184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a98204-ac44-4492-ae30-e25434d8f5fc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:45.647660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 193/383] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:45 2022] GET /identity/v3/role_assignments?group.id=604ee7930c4b48c486e6532caf56a94e&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=7583ad887d8b4052bd11783371c338b4 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:45.790680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd17818-e41d-4706-ad24-9d033a44954a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:45.847509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da911c9-7646-4aa1-9b04-a37426862718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:45.848217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da911c9-7646-4aa1-9b04-a37426862718 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:45.848501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da911c9-7646-4aa1-9b04-a37426862718 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:45.848778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da911c9-7646-4aa1-9b04-a37426862718 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:45.849624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da911c9-7646-4aa1-9b04-a37426862718 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:45.849955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da911c9-7646-4aa1-9b04-a37426862718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:45.851631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da911c9-7646-4aa1-9b04-a37426862718 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:45.866637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 191/384] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:45.872719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a98204-ac44-4492-ae30-e25434d8f5fc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:45.880744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:45.997030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a98204-ac44-4492-ae30-e25434d8f5fc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:46.044268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71ca25-e573-4e09-b4d8-2807c0215b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:46.045521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71ca25-e573-4e09-b4d8-2807c0215b27 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:46.046025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71ca25-e573-4e09-b4d8-2807c0215b27 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:46.046510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71ca25-e573-4e09-b4d8-2807c0215b27 None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:46.048068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71ca25-e573-4e09-b4d8-2807c0215b27 None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:46.049695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71ca25-e573-4e09-b4d8-2807c0215b27 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:46.051859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 194/385] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:45 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:46.057621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da911c9-7646-4aa1-9b04-a37426862718 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:46.154997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:46.202429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da911c9-7646-4aa1-9b04-a37426862718 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:46.245602 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901e093-e1f4-4d43-944e-b85cdb37cd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:46.246501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901e093-e1f4-4d43-944e-b85cdb37cd3b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:46.246790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901e093-e1f4-4d43-944e-b85cdb37cd3b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:46.247125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901e093-e1f4-4d43-944e-b85cdb37cd3b None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:46.248285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901e093-e1f4-4d43-944e-b85cdb37cd3b None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:46.249756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901e093-e1f4-4d43-944e-b85cdb37cd3b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:46.252463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 192/386] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:46 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:46.257094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71ca25-e573-4e09-b4d8-2807c0215b27 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:46.366056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:46.402928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71ca25-e573-4e09-b4d8-2807c0215b27 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:46.445445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fdd37-63b3-4e49-b2ca-7546baac70b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:46.446370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fdd37-63b3-4e49-b2ca-7546baac70b1 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:46.446655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fdd37-63b3-4e49-b2ca-7546baac70b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:46.446936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fdd37-63b3-4e49-b2ca-7546baac70b1 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7/groups/604ee7930c4b48c486e6532caf56a94e/roles/7583ad887d8b4052bd11783371c338b4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:46.451713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fdd37-63b3-4e49-b2ca-7546baac70b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca796ca2de804171942d2631cccd56c7, group_id=604ee7930c4b48c486e6532caf56a94e, role_id=7583ad887d8b4052bd11783371c338b4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:46.453179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fdd37-63b3-4e49-b2ca-7546baac70b1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:46.492914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c671cb5-5377-4ba9-8b24-6aa42b0b6d39 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:46.493264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c671cb5-5377-4ba9-8b24-6aa42b0b6d39 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:46.493479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c671cb5-5377-4ba9-8b24-6aa42b0b6d39 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:46.495161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 193/387] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:46.500707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abfe08-b032-4de3-9e32-66dd34e000d6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:46.500948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abfe08-b032-4de3-9e32-66dd34e000d6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:46.501232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abfe08-b032-4de3-9e32-66dd34e000d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:46.518916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d2abfe08-b032-4de3-9e32-66dd34e000d6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:46.522068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d2abfe08-b032-4de3-9e32-66dd34e000d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:46.528448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 195/388] 10.222.0.42 () {58 vars in 1442 bytes} [Fri Jun 3 09:37:46 2022] PUT /identity/v3/projects/ca796ca2de804171942d2631cccd56c7/groups/604ee7930c4b48c486e6532caf56a94e/roles/7583ad887d8b4052bd11783371c338b4 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:46.623159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 194/389] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:46.667462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fdd37-63b3-4e49-b2ca-7546baac70b1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:46.793787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fdd37-63b3-4e49-b2ca-7546baac70b1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:46.829549 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:46.835005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935f9fc-193f-4da0-9398-794ed7e50f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:46.836441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935f9fc-193f-4da0-9398-794ed7e50f9f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:46.836729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935f9fc-193f-4da0-9398-794ed7e50f9f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:46.837060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935f9fc-193f-4da0-9398-794ed7e50f9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:46.848539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935f9fc-193f-4da0-9398-794ed7e50f9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-4935f9fc-193f-4da0-9398-794ed7e50f9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:46.857121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:46.858013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:46.858773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 196/390] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:46.864287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abfe08-b032-4de3-9e32-66dd34e000d6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:46.956436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:47.010461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abfe08-b032-4de3-9e32-66dd34e000d6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:47.063662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7da064-9bb1-4ab9-ad1b-6d68e5887509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:47.064266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7da064-9bb1-4ab9-ad1b-6d68e5887509 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:47.064552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7da064-9bb1-4ab9-ad1b-6d68e5887509 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:47.064828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7da064-9bb1-4ab9-ad1b-6d68e5887509 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:47.065584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7da064-9bb1-4ab9-ad1b-6d68e5887509 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:47.065909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7da064-9bb1-4ab9-ad1b-6d68e5887509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:47.067293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7da064-9bb1-4ab9-ad1b-6d68e5887509 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:47.081611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 195/391] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:47.087333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935f9fc-193f-4da0-9398-794ed7e50f9f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:47.254417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935f9fc-193f-4da0-9398-794ed7e50f9f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:47.305960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c353d91-be49-4780-b658-307d3401bb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:47.307185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c353d91-be49-4780-b658-307d3401bb80 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:47.307649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c353d91-be49-4780-b658-307d3401bb80 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:47.308205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c353d91-be49-4780-b658-307d3401bb80 None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:47.311941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c353d91-be49-4780-b658-307d3401bb80 None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:47.313735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c353d91-be49-4780-b658-307d3401bb80 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:47.315709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 197/392] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:47 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:47.321721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7da064-9bb1-4ab9-ad1b-6d68e5887509 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:47.445370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ec8f5-61b2-41ad-85a6-fa67ec9ccf10 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:47.445661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ec8f5-61b2-41ad-85a6-fa67ec9ccf10 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:47.445983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ec8f5-61b2-41ad-85a6-fa67ec9ccf10 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:47.448454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 198/393] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:47.453482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15b62b-78c4-4799-be88-d46c74ae7d0d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:47.453774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15b62b-78c4-4799-be88-d46c74ae7d0d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:47.454097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15b62b-78c4-4799-be88-d46c74ae7d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:47.467501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7da064-9bb1-4ab9-ad1b-6d68e5887509 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:47.482599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7b15b62b-78c4-4799-be88-d46c74ae7d0d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:47.483784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7b15b62b-78c4-4799-be88-d46c74ae7d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:47.499971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:47.524114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0d3fa-9fe6-403e-a985-8a32e595754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:47.524976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0d3fa-9fe6-403e-a985-8a32e595754d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:47.525254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0d3fa-9fe6-403e-a985-8a32e595754d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:47.525528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0d3fa-9fe6-403e-a985-8a32e595754d None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:47.527684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0d3fa-9fe6-403e-a985-8a32e595754d None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:47.529151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0d3fa-9fe6-403e-a985-8a32e595754d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:47.532535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 196/394] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:47 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:47.537225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0d3fa-9fe6-403e-a985-8a32e595754d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:47.586661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:47.612543 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 199/395] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:47.658534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15b62b-78c4-4799-be88-d46c74ae7d0d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:47.695353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0d3fa-9fe6-403e-a985-8a32e595754d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:47.754230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec19cd-905a-4c9e-aadd-b81142e7a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:47.755123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec19cd-905a-4c9e-aadd-b81142e7a0cf None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:47.755461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec19cd-905a-4c9e-aadd-b81142e7a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:47.755743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec19cd-905a-4c9e-aadd-b81142e7a0cf None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7/groups/604ee7930c4b48c486e6532caf56a94e/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:47.758609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec19cd-905a-4c9e-aadd-b81142e7a0cf None admin] RBAC: Authorizing `identity:create_grant(project_id=ca796ca2de804171942d2631cccd56c7, group_id=604ee7930c4b48c486e6532caf56a94e, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:47.759937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec19cd-905a-4c9e-aadd-b81142e7a0cf None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:47.784951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15b62b-78c4-4799-be88-d46c74ae7d0d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:47.822798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3f1e1-0b8d-4fba-b405-2b42720d599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:47.823990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3f1e1-0b8d-4fba-b405-2b42720d599c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:47.824496 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3f1e1-0b8d-4fba-b405-2b42720d599c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:47.824975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3f1e1-0b8d-4fba-b405-2b42720d599c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:47.836017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3f1e1-0b8d-4fba-b405-2b42720d599c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:47.841834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 197/396] 10.222.0.42 () {58 vars in 1442 bytes} [Fri Jun 3 09:37:47 2022] PUT /identity/v3/projects/ca796ca2de804171942d2631cccd56c7/groups/604ee7930c4b48c486e6532caf56a94e/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-00f3f1e1-0b8d-4fba-b405-2b42720d599c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:47.845169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:47.846284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:47.847325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:47.849651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 200/397] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:47.856556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec19cd-905a-4c9e-aadd-b81142e7a0cf None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:47.918927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:47.986544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec19cd-905a-4c9e-aadd-b81142e7a0cf None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:48.034164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb599a-c6c4-4d2b-be42-9fc0cd747465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:48.035153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb599a-c6c4-4d2b-be42-9fc0cd747465 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:48.035583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb599a-c6c4-4d2b-be42-9fc0cd747465 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:48.036032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb599a-c6c4-4d2b-be42-9fc0cd747465 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:48.036926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb599a-c6c4-4d2b-be42-9fc0cd747465 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:48.037389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb599a-c6c4-4d2b-be42-9fc0cd747465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:48.039184 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb599a-c6c4-4d2b-be42-9fc0cd747465 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:48.055270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 198/398] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:48.061553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3f1e1-0b8d-4fba-b405-2b42720d599c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:48.124366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:48.198571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3f1e1-0b8d-4fba-b405-2b42720d599c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:48.240582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6e1f2-8e6b-4ca9-97e8-1f36b4133403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:48.241454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6e1f2-8e6b-4ca9-97e8-1f36b4133403 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:48.242279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6e1f2-8e6b-4ca9-97e8-1f36b4133403 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:48.242563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6e1f2-8e6b-4ca9-97e8-1f36b4133403 None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:48.243779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6e1f2-8e6b-4ca9-97e8-1f36b4133403 None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:48.245246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6e1f2-8e6b-4ca9-97e8-1f36b4133403 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:48.251418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 201/399] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:48 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:48.252779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb599a-c6c4-4d2b-be42-9fc0cd747465 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:48.357353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb599a-c6c4-4d2b-be42-9fc0cd747465 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:48.368540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:48.406390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee381c1-906b-488e-80d9-f796684b8150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:48.407185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee381c1-906b-488e-80d9-f796684b8150 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:48.407476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee381c1-906b-488e-80d9-f796684b8150 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:48.407813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee381c1-906b-488e-80d9-f796684b8150 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:48.409464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee381c1-906b-488e-80d9-f796684b8150 None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:48.410946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee381c1-906b-488e-80d9-f796684b8150 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:48.413986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 199/400] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:48 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:48.418996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6e1f2-8e6b-4ca9-97e8-1f36b4133403 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:48.521806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6e1f2-8e6b-4ca9-97e8-1f36b4133403 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:48.538075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:48.544729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10683969-699c-4b80-a4b8-792882447294 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:48.545017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10683969-699c-4b80-a4b8-792882447294 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:48.545303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10683969-699c-4b80-a4b8-792882447294 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:48.547013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 200/401] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:48.552180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180af5e-fa8d-4d47-8d17-867f9bdb4f1a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:48.552472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180af5e-fa8d-4d47-8d17-867f9bdb4f1a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:48.552754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180af5e-fa8d-4d47-8d17-867f9bdb4f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:48.568782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b92f3-d2a2-4d9b-b4f0-a22070582f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:48.569670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b92f3-d2a2-4d9b-b4f0-a22070582f43 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:48.570565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b92f3-d2a2-4d9b-b4f0-a22070582f43 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:48.570847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b92f3-d2a2-4d9b-b4f0-a22070582f43 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986/groups/604ee7930c4b48c486e6532caf56a94e/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:48.575827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b92f3-d2a2-4d9b-b4f0-a22070582f43 None admin] RBAC: Authorizing `identity:create_grant(project_id=a91190fa89bd41f49048e6128973a986, group_id=604ee7930c4b48c486e6532caf56a94e, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:48.577233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b92f3-d2a2-4d9b-b4f0-a22070582f43 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:48.580550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6180af5e-fa8d-4d47-8d17-867f9bdb4f1a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:48.581781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6180af5e-fa8d-4d47-8d17-867f9bdb4f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:48.651071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 201/402] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:48.652399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e903b9-7858-4886-8b62-59d53f61263d None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:48.652687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e903b9-7858-4886-8b62-59d53f61263d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:48.652967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e903b9-7858-4886-8b62-59d53f61263d None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:48.654341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 202/403] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:48.659878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c37e4-873c-49c8-937a-d78103bf69a2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:48.660109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c37e4-873c-49c8-937a-d78103bf69a2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:48.660417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c37e4-873c-49c8-937a-d78103bf69a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:48.666769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 202/404] 10.222.0.42 () {58 vars in 1442 bytes} [Fri Jun 3 09:37:48 2022] PUT /identity/v3/projects/a91190fa89bd41f49048e6128973a986/groups/604ee7930c4b48c486e6532caf56a94e/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:48.685826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b92f3-d2a2-4d9b-b4f0-a22070582f43 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:48.691183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fc9c37e4-873c-49c8-937a-d78103bf69a2 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:48.692389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fc9c37e4-873c-49c8-937a-d78103bf69a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:48.778903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 203/405] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:48.803281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c37e4-873c-49c8-937a-d78103bf69a2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:48.884888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:48.905805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b92f3-d2a2-4d9b-b4f0-a22070582f43 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:48.941222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f0b5-965d-4fe9-800d-b0a3d03b2ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:48.941978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f0b5-965d-4fe9-800d-b0a3d03b2ec5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:48.942268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f0b5-965d-4fe9-800d-b0a3d03b2ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:48.942555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f0b5-965d-4fe9-800d-b0a3d03b2ec5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:48.953301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4f0b5-965d-4fe9-800d-b0a3d03b2ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:48.960127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c37e4-873c-49c8-937a-d78103bf69a2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9bc4f0b5-965d-4fe9-800d-b0a3d03b2ec5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:48.961692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:48.962710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:48.963490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 203/406] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:37:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:48.970844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f0b5-965d-4fe9-800d-b0a3d03b2ec5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:49.002670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659a8e4-f897-4fe9-a03a-908728f6d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:49.003978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659a8e4-f897-4fe9-a03a-908728f6d463 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.004502 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659a8e4-f897-4fe9-a03a-908728f6d463 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.004976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659a8e4-f897-4fe9-a03a-908728f6d463 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.011161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:49.017412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659a8e4-f897-4fe9-a03a-908728f6d463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f659a8e4-f897-4fe9-a03a-908728f6d463 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:49.028208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:49.029103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:49.029924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:49.032614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 204/407] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:49.038665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659a8e4-f897-4fe9-a03a-908728f6d463 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:49.081375 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f0b5-965d-4fe9-800d-b0a3d03b2ec5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:49.120438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf57b6-8e91-44d8-a7da-c9f65071e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:49.121106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf57b6-8e91-44d8-a7da-c9f65071e888 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.121386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf57b6-8e91-44d8-a7da-c9f65071e888 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.121658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf57b6-8e91-44d8-a7da-c9f65071e888 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.122437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf57b6-8e91-44d8-a7da-c9f65071e888 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:49.122740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf57b6-8e91-44d8-a7da-c9f65071e888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:49.124194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf57b6-8e91-44d8-a7da-c9f65071e888 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:49.133131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 204/408] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:49.139814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf57b6-8e91-44d8-a7da-c9f65071e888 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:49.220352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:49.224505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659a8e4-f897-4fe9-a03a-908728f6d463 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:49.267794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76857c4-9b4b-4886-989f-a23e85bba1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:49.268470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76857c4-9b4b-4886-989f-a23e85bba1bc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.268752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76857c4-9b4b-4886-989f-a23e85bba1bc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.269027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76857c4-9b4b-4886-989f-a23e85bba1bc None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.269826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76857c4-9b4b-4886-989f-a23e85bba1bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:49.270157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76857c4-9b4b-4886-989f-a23e85bba1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:49.271547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76857c4-9b4b-4886-989f-a23e85bba1bc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:49.288286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 205/409] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:49.294812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76857c4-9b4b-4886-989f-a23e85bba1bc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:49.369639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:49.376511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf57b6-8e91-44d8-a7da-c9f65071e888 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:49.452665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db10c8-eb5d-4b6c-ab16-5d8ba091c1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:49.453538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db10c8-eb5d-4b6c-ab16-5d8ba091c1cf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.453820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db10c8-eb5d-4b6c-ab16-5d8ba091c1cf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.454130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db10c8-eb5d-4b6c-ab16-5d8ba091c1cf None admin] PATH_INFO: `/v3/groups/51186006fd854cbc846ca13ba62d9289` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.456299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db10c8-eb5d-4b6c-ab16-5d8ba091c1cf None admin] RBAC: Authorizing `identity:get_group(group_id=51186006fd854cbc846ca13ba62d9289)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:49.457747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db10c8-eb5d-4b6c-ab16-5d8ba091c1cf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:49.459664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 205/410] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:49 2022] GET /identity/v3/groups/51186006fd854cbc846ca13ba62d9289 => generated 229 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:49.460932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4e51-36b6-4293-a216-23f960f4134b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.461218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4e51-36b6-4293-a216-23f960f4134b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.461497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4e51-36b6-4293-a216-23f960f4134b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.469525 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 206/411] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:49.469525 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4e51-36b6-4293-a216-23f960f4134b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:49.479400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76857c4-9b4b-4886-989f-a23e85bba1bc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:49.520999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a74bb-3cf9-49cc-bad9-b13c078f2a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:49.522656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a74bb-3cf9-49cc-bad9-b13c078f2a8a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.523280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a74bb-3cf9-49cc-bad9-b13c078f2a8a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.523758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a74bb-3cf9-49cc-bad9-b13c078f2a8a None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.525819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a74bb-3cf9-49cc-bad9-b13c078f2a8a None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:49.527858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a74bb-3cf9-49cc-bad9-b13c078f2a8a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:49.530465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 206/412] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:49 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:49.532140 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ecead-0030-421e-8e48-b1a25a6e0fff None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.532910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ecead-0030-421e-8e48-b1a25a6e0fff None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.533394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ecead-0030-421e-8e48-b1a25a6e0fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.557288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-508ecead-0030-421e-8e48-b1a25a6e0fff None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:49.559080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-508ecead-0030-421e-8e48-b1a25a6e0fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:49.583235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:49.592302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4e51-36b6-4293-a216-23f960f4134b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:49.634529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7bf7a-166f-48f1-94bc-f8b3e55ee99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:49.635778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7bf7a-166f-48f1-94bc-f8b3e55ee99c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.636260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7bf7a-166f-48f1-94bc-f8b3e55ee99c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.636759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7bf7a-166f-48f1-94bc-f8b3e55ee99c None admin] PATH_INFO: `/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.638003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 207/413] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:49.638386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7bf7a-166f-48f1-94bc-f8b3e55ee99c None admin] RBAC: Authorizing `identity:get_project(project_id=53d2a61c9fa84503ab8a6c4eb2d3f27e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:49.638940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ecead-0030-421e-8e48-b1a25a6e0fff None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:49.640250 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7bf7a-166f-48f1-94bc-f8b3e55ee99c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:49.642670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 207/414] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:49 2022] GET /identity/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:49.649759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7bf7a-166f-48f1-94bc-f8b3e55ee99c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:49.690197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:49.764994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ecead-0030-421e-8e48-b1a25a6e0fff None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:49.774263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7bf7a-166f-48f1-94bc-f8b3e55ee99c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:49.800902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61c977-4e5c-42c5-b383-8993b6d285f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:49.801812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61c977-4e5c-42c5-b383-8993b6d285f5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.802144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61c977-4e5c-42c5-b383-8993b6d285f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.802424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61c977-4e5c-42c5-b383-8993b6d285f5 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.803706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61c977-4e5c-42c5-b383-8993b6d285f5 None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:49.805781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61c977-4e5c-42c5-b383-8993b6d285f5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:49.807570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 208/415] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:49 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:49.808615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61c977-4e5c-42c5-b383-8993b6d285f5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:49.817210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4a7c-c5d5-4389-9c8a-ed21ac6ba6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:49.818758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4a7c-c5d5-4389-9c8a-ed21ac6ba6d3 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.819269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4a7c-c5d5-4389-9c8a-ed21ac6ba6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.820175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4a7c-c5d5-4389-9c8a-ed21ac6ba6d3 None admin] PATH_INFO: `/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e/groups/51186006fd854cbc846ca13ba62d9289/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:49.822909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff4a7c-c5d5-4389-9c8a-ed21ac6ba6d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=53d2a61c9fa84503ab8a6c4eb2d3f27e, group_id=51186006fd854cbc846ca13ba62d9289, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:49.825057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff4a7c-c5d5-4389-9c8a-ed21ac6ba6d3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:49.837626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:49.902606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 208/416] 10.222.0.42 () {58 vars in 1442 bytes} [Fri Jun 3 09:37:49 2022] PUT /identity/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e/groups/51186006fd854cbc846ca13ba62d9289/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:37:49.904267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4a7c-c5d5-4389-9c8a-ed21ac6ba6d3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:49.923580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61c977-4e5c-42c5-b383-8993b6d285f5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:49.988283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953cd59-a0b2-43f3-8bf8-6d0ffa018c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:49.989111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953cd59-a0b2-43f3-8bf8-6d0ffa018c52 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:49.989398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953cd59-a0b2-43f3-8bf8-6d0ffa018c52 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:49.989678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953cd59-a0b2-43f3-8bf8-6d0ffa018c52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:50.019108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953cd59-a0b2-43f3-8bf8-6d0ffa018c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-2953cd59-a0b2-43f3-8bf8-6d0ffa018c52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:50.031987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:50.032919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:50.033679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:50.038254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 209/417] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:50.040619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953cd59-a0b2-43f3-8bf8-6d0ffa018c52 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:50.135190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:50.149261 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4a7c-c5d5-4389-9c8a-ed21ac6ba6d3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:50.205472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b929b7-3330-409f-9c1c-8fb1f5305e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:50.206411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b929b7-3330-409f-9c1c-8fb1f5305e37 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:50.206699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b929b7-3330-409f-9c1c-8fb1f5305e37 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:50.207041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b929b7-3330-409f-9c1c-8fb1f5305e37 None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986/groups/604ee7930c4b48c486e6532caf56a94e/roles/7583ad887d8b4052bd11783371c338b4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:50.209275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:50.209385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b929b7-3330-409f-9c1c-8fb1f5305e37 None admin] RBAC: Authorizing `identity:create_grant(project_id=a91190fa89bd41f49048e6128973a986, group_id=604ee7930c4b48c486e6532caf56a94e, role_id=7583ad887d8b4052bd11783371c338b4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:50.211012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b929b7-3330-409f-9c1c-8fb1f5305e37 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:50.233619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953cd59-a0b2-43f3-8bf8-6d0ffa018c52 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:50.270057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a6002-e58f-4122-9704-6c8a6bb8835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:50.270652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a6002-e58f-4122-9704-6c8a6bb8835c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:50.271024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a6002-e58f-4122-9704-6c8a6bb8835c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:50.271299 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a6002-e58f-4122-9704-6c8a6bb8835c None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:50.272044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a6002-e58f-4122-9704-6c8a6bb8835c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:50.272345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a6002-e58f-4122-9704-6c8a6bb8835c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:50.273764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a6002-e58f-4122-9704-6c8a6bb8835c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:50.282333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 210/418] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:50.286045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 209/419] 10.222.0.42 () {58 vars in 1442 bytes} [Fri Jun 3 09:37:49 2022] PUT /identity/v3/projects/a91190fa89bd41f49048e6128973a986/groups/604ee7930c4b48c486e6532caf56a94e/roles/7583ad887d8b4052bd11783371c338b4 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:37:50.288056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a6002-e58f-4122-9704-6c8a6bb8835c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:50.403997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a6002-e58f-4122-9704-6c8a6bb8835c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:50.440432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75542a8-1197-4608-ab6e-4f87b6cfec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:50.441082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75542a8-1197-4608-ab6e-4f87b6cfec38 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:50.441363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75542a8-1197-4608-ab6e-4f87b6cfec38 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:50.441640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75542a8-1197-4608-ab6e-4f87b6cfec38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:50.461969 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:50.515938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75542a8-1197-4608-ab6e-4f87b6cfec38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:50.517462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75542a8-1197-4608-ab6e-4f87b6cfec38 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:50.639119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 211/420] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 09:37:50.644868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b929b7-3330-409f-9c1c-8fb1f5305e37 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:50.663444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebda7c-ddb8-4bf6-be57-2aa9ecd266b4 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:50.664671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebda7c-ddb8-4bf6-be57-2aa9ecd266b4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:50.667423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebda7c-ddb8-4bf6-be57-2aa9ecd266b4 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:50.670736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 212/421] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:50.674713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8f5d0-d050-4816-a2ed-5b4622958794 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:50.675237 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8f5d0-d050-4816-a2ed-5b4622958794 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:50.675942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8f5d0-d050-4816-a2ed-5b4622958794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:50.703921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c2e8f5d0-d050-4816-a2ed-5b4622958794 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:50.705567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c2e8f5d0-d050-4816-a2ed-5b4622958794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:50.766683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:50.787641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b929b7-3330-409f-9c1c-8fb1f5305e37 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:50.795044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 213/422] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:50.823021 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8f5d0-d050-4816-a2ed-5b4622958794 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:50.844238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29dabc-1b37-4f99-a96e-1b332ed3d177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:50.846160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29dabc-1b37-4f99-a96e-1b332ed3d177 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:50.846832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29dabc-1b37-4f99-a96e-1b332ed3d177 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:50.848698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29dabc-1b37-4f99-a96e-1b332ed3d177 None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:50.850173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29dabc-1b37-4f99-a96e-1b332ed3d177 None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:50.852645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29dabc-1b37-4f99-a96e-1b332ed3d177 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:50.857127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 210/423] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:50 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:50.866999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29dabc-1b37-4f99-a96e-1b332ed3d177 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:50.913690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:50.943707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8f5d0-d050-4816-a2ed-5b4622958794 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:50.979583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062df203-443e-4603-98ad-8488482e62d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:50.980381 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062df203-443e-4603-98ad-8488482e62d7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:50.980660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062df203-443e-4603-98ad-8488482e62d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:50.980937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062df203-443e-4603-98ad-8488482e62d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:50.992438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062df203-443e-4603-98ad-8488482e62d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-062df203-443e-4603-98ad-8488482e62d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:51.000308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:51.001265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:51.002322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 214/424] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:51.009399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062df203-443e-4603-98ad-8488482e62d7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:51.055286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:51.071737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29dabc-1b37-4f99-a96e-1b332ed3d177 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:51.140006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8fb37-257d-4d07-8afe-e34d83d6a4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:51.141530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8fb37-257d-4d07-8afe-e34d83d6a4f8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:51.142015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8fb37-257d-4d07-8afe-e34d83d6a4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:51.142437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8fb37-257d-4d07-8afe-e34d83d6a4f8 None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:51.143761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8fb37-257d-4d07-8afe-e34d83d6a4f8 None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:51.145354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8fb37-257d-4d07-8afe-e34d83d6a4f8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:51.147514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 211/425] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:50 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:51.153535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8fb37-257d-4d07-8afe-e34d83d6a4f8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:51.169837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062df203-443e-4603-98ad-8488482e62d7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:51.236189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fabe3-7b87-42f7-a00e-75b6fbad2d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:51.237013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fabe3-7b87-42f7-a00e-75b6fbad2d68 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:51.237393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fabe3-7b87-42f7-a00e-75b6fbad2d68 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:51.237764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fabe3-7b87-42f7-a00e-75b6fbad2d68 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:51.238679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fabe3-7b87-42f7-a00e-75b6fbad2d68 None admin] RBAC: Adding query filter params (name=member) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:51.239084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fabe3-7b87-42f7-a00e-75b6fbad2d68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:51.240760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fabe3-7b87-42f7-a00e-75b6fbad2d68 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:51.253738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 215/426] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:51.258807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fabe3-7b87-42f7-a00e-75b6fbad2d68 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:51.293433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:51.305044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8fb37-257d-4d07-8afe-e34d83d6a4f8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:51.343683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681a5d-31af-4fa5-a5e2-e35989088459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:51.344341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681a5d-31af-4fa5-a5e2-e35989088459 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:51.344620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681a5d-31af-4fa5-a5e2-e35989088459 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:51.344893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681a5d-31af-4fa5-a5e2-e35989088459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:51.345719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc681a5d-31af-4fa5-a5e2-e35989088459 None admin] RBAC: Adding query filter params (group.id=604ee7930c4b48c486e6532caf56a94e, role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=ca796ca2de804171942d2631cccd56c7) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:51.346061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc681a5d-31af-4fa5-a5e2-e35989088459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:51.348005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc681a5d-31af-4fa5-a5e2-e35989088459 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:51.355343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 212/427] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:51 2022] GET /identity/v3/role_assignments?group.id=604ee7930c4b48c486e6532caf56a94e&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=7583ad887d8b4052bd11783371c338b4 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:51.402778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a6177-0af1-4a17-b69c-b3b6fdd7dec6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:51.403097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a6177-0af1-4a17-b69c-b3b6fdd7dec6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:51.403344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a6177-0af1-4a17-b69c-b3b6fdd7dec6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:51.404980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 213/428] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:51.410241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ce98-762f-4a2f-953f-de9e9ecfa401 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:51.410530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ce98-762f-4a2f-953f-de9e9ecfa401 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:51.410814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ce98-762f-4a2f-953f-de9e9ecfa401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:51.452710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0a17ce98-762f-4a2f-953f-de9e9ecfa401 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:51.452710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0a17ce98-762f-4a2f-953f-de9e9ecfa401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:51.460115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fabe3-7b87-42f7-a00e-75b6fbad2d68 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:51.473293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:51.520601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ce8f3-7327-4957-ad48-5ecee6a09f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:51.521788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ce8f3-7327-4957-ad48-5ecee6a09f43 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:51.522228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ce8f3-7327-4957-ad48-5ecee6a09f43 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:51.522509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ce8f3-7327-4957-ad48-5ecee6a09f43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:51.598925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ce8f3-7327-4957-ad48-5ecee6a09f43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:51.600549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ce8f3-7327-4957-ad48-5ecee6a09f43 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:51.628802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 214/429] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:51.658117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17ce98-762f-4a2f-953f-de9e9ecfa401 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:51.702671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:51.706671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 216/430] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:51.712369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ce8f3-7327-4957-ad48-5ecee6a09f43 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:51.771265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:51.789278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17ce98-762f-4a2f-953f-de9e9ecfa401 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:51.832229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ff110-6b57-4a3b-877b-5fb261317eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:51.833022 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ff110-6b57-4a3b-877b-5fb261317eaa None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:51.833307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ff110-6b57-4a3b-877b-5fb261317eaa None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:51.833637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ff110-6b57-4a3b-877b-5fb261317eaa None admin] PATH_INFO: `/v3/roles/member` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:51.844394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ce8f3-7327-4957-ad48-5ecee6a09f43 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:51.848861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ff110-6b57-4a3b-877b-5fb261317eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-7c3ff110-6b57-4a3b-877b-5fb261317eaa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:51.857443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:51.858529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 09:37:51.859392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:51.866057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 215/431] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:51.867731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ff110-6b57-4a3b-877b-5fb261317eaa None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:51.888825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecea92-99e4-4d8e-a65c-8971e953241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:51.890318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecea92-99e4-4d8e-a65c-8971e953241d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:51.891167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecea92-99e4-4d8e-a65c-8971e953241d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:51.891642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecea92-99e4-4d8e-a65c-8971e953241d None admin] PATH_INFO: `/v3/projects/ca796ca2de804171942d2631cccd56c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:51.893030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecea92-99e4-4d8e-a65c-8971e953241d None admin] RBAC: Authorizing `identity:get_project(project_id=ca796ca2de804171942d2631cccd56c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:51.895093 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecea92-99e4-4d8e-a65c-8971e953241d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:51.897023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 217/432] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:51 2022] GET /identity/v3/projects/ca796ca2de804171942d2631cccd56c7 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:51.903734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecea92-99e4-4d8e-a65c-8971e953241d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:51.978561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:52.002100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ff110-6b57-4a3b-877b-5fb261317eaa None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:52.013536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecea92-99e4-4d8e-a65c-8971e953241d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:52.039186 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71d143-6ca4-48a1-a770-c7f4ee421bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:52.040192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71d143-6ca4-48a1-a770-c7f4ee421bca None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:52.040923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71d143-6ca4-48a1-a770-c7f4ee421bca None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:52.041407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71d143-6ca4-48a1-a770-c7f4ee421bca None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:52.042388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71d143-6ca4-48a1-a770-c7f4ee421bca None admin] RBAC: Adding query filter params (name=member) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:52.043008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71d143-6ca4-48a1-a770-c7f4ee421bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:52.044573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71d143-6ca4-48a1-a770-c7f4ee421bca None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:52.050494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cea1a-c5d5-4eed-8eb5-b57466a4d009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:52.051360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cea1a-c5d5-4eed-8eb5-b57466a4d009 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:52.051646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cea1a-c5d5-4eed-8eb5-b57466a4d009 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:52.051925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cea1a-c5d5-4eed-8eb5-b57466a4d009 None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:52.053151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cea1a-c5d5-4eed-8eb5-b57466a4d009 None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:52.055342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cea1a-c5d5-4eed-8eb5-b57466a4d009 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:52.055572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 216/433] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:37:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:52.059524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 218/434] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:51 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:52.065218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71d143-6ca4-48a1-a770-c7f4ee421bca None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:52.065369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cea1a-c5d5-4eed-8eb5-b57466a4d009 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:52.073609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:52.166331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71d143-6ca4-48a1-a770-c7f4ee421bca None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:52.173630 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cea1a-c5d5-4eed-8eb5-b57466a4d009 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:52.204853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f4119-ec93-42f2-965a-fc9c66c3ac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:52.205806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f4119-ec93-42f2-965a-fc9c66c3ac7b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:52.206338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f4119-ec93-42f2-965a-fc9c66c3ac7b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:52.206759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f4119-ec93-42f2-965a-fc9c66c3ac7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:52.213905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e546c-a24f-4ef3-9679-7710901ecdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:52.214657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e546c-a24f-4ef3-9679-7710901ecdc9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:52.215014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e546c-a24f-4ef3-9679-7710901ecdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:52.215367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e546c-a24f-4ef3-9679-7710901ecdc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:52.216274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e546c-a24f-4ef3-9679-7710901ecdc9 None admin] RBAC: Adding query filter params (group.id=604ee7930c4b48c486e6532caf56a94e, role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=ca796ca2de804171942d2631cccd56c7) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:52.216656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e546c-a24f-4ef3-9679-7710901ecdc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:52.218218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e546c-a24f-4ef3-9679-7710901ecdc9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:52.221277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:52.226445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 219/435] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:52 2022] GET /identity/v3/role_assignments?group.id=604ee7930c4b48c486e6532caf56a94e&scope.project.id=ca796ca2de804171942d2631cccd56c7&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:52.297983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f4119-ec93-42f2-965a-fc9c66c3ac7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:52.299487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f4119-ec93-42f2-965a-fc9c66c3ac7b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:52.309635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:52.525290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829a213-736a-45e5-9683-5a05b3825997 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:52.525728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829a213-736a-45e5-9683-5a05b3825997 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:52.526070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829a213-736a-45e5-9683-5a05b3825997 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:52.527711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 220/436] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:37:52.532293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 217/437] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 09:37:52.533056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3b908-ce3d-4a10-bef5-1438a324f2cf None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:52.533343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3b908-ce3d-4a10-bef5-1438a324f2cf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:52.533618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3b908-ce3d-4a10-bef5-1438a324f2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:52.537813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f4119-ec93-42f2-965a-fc9c66c3ac7b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:52.552361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d1b3b908-ce3d-4a10-bef5-1438a324f2cf None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:52.553526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d1b3b908-ce3d-4a10-bef5-1438a324f2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:52.633815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 221/438] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:52.650295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f4119-ec93-42f2-965a-fc9c66c3ac7b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:52.662416 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:52.662833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3b908-ce3d-4a10-bef5-1438a324f2cf None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:52.700944 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a059-d21e-4d33-b837-0c47022645fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:52.701803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a059-d21e-4d33-b837-0c47022645fa None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:52.702114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a059-d21e-4d33-b837-0c47022645fa None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:52.702385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a059-d21e-4d33-b837-0c47022645fa None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:52.703592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1a059-d21e-4d33-b837-0c47022645fa None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:52.705043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1a059-d21e-4d33-b837-0c47022645fa None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:52.706864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 218/439] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:52 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:52.720520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a059-d21e-4d33-b837-0c47022645fa None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:52.796455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3b908-ce3d-4a10-bef5-1438a324f2cf None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:52.840685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:52.842532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189cb9e3-797f-422e-a959-b2eeffad4434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:52.843334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cb9e3-797f-422e-a959-b2eeffad4434 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:52.843878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cb9e3-797f-422e-a959-b2eeffad4434 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:52.844200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189cb9e3-797f-422e-a959-b2eeffad4434 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:52.850344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a059-d21e-4d33-b837-0c47022645fa None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:52.859928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189cb9e3-797f-422e-a959-b2eeffad4434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-189cb9e3-797f-422e-a959-b2eeffad4434 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:52.869321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:52.870283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:52.871056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 222/440] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:52.878509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189cb9e3-797f-422e-a959-b2eeffad4434 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:52.924455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7157a3-5dbc-4dc1-9599-ad382825ae29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:52.925332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7157a3-5dbc-4dc1-9599-ad382825ae29 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:52.925636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7157a3-5dbc-4dc1-9599-ad382825ae29 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:52.925932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7157a3-5dbc-4dc1-9599-ad382825ae29 None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:52.929497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7157a3-5dbc-4dc1-9599-ad382825ae29 None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:52.931060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7157a3-5dbc-4dc1-9599-ad382825ae29 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:52.932844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 219/441] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:52 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:52.938478 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7157a3-5dbc-4dc1-9599-ad382825ae29 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:52.995454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:53.008821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189cb9e3-797f-422e-a959-b2eeffad4434 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:53.048665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea02244-4b20-45c0-9880-fcf8d24194e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:53.049373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea02244-4b20-45c0-9880-fcf8d24194e5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.049664 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea02244-4b20-45c0-9880-fcf8d24194e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.049969 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea02244-4b20-45c0-9880-fcf8d24194e5 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.050715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea02244-4b20-45c0-9880-fcf8d24194e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:53.051023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea02244-4b20-45c0-9880-fcf8d24194e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:53.052427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea02244-4b20-45c0-9880-fcf8d24194e5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:53.055187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7157a3-5dbc-4dc1-9599-ad382825ae29 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:53.065403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 223/442] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:37:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:53.066443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea02244-4b20-45c0-9880-fcf8d24194e5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:53.094260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946b2d5-99cd-46f6-9853-764cd413ec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:53.094828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b2d5-99cd-46f6-9853-764cd413ec46 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.095116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b2d5-99cd-46f6-9853-764cd413ec46 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.095399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b2d5-99cd-46f6-9853-764cd413ec46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.096229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946b2d5-99cd-46f6-9853-764cd413ec46 None admin] RBAC: Adding query filter params (group.id=604ee7930c4b48c486e6532caf56a94e, role.id=7bcc1da1e79942c0a21920d35347bf06, scope.project.id=a91190fa89bd41f49048e6128973a986) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:53.096547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946b2d5-99cd-46f6-9853-764cd413ec46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:53.098039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946b2d5-99cd-46f6-9853-764cd413ec46 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:53.104310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 220/443] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:52 2022] GET /identity/v3/role_assignments?group.id=604ee7930c4b48c486e6532caf56a94e&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=7bcc1da1e79942c0a21920d35347bf06 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:53.116513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c1e6a-4de4-45ec-a4d0-baac57c99a75 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.117118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c1e6a-4de4-45ec-a4d0-baac57c99a75 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.117411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c1e6a-4de4-45ec-a4d0-baac57c99a75 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.119259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 221/444] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:53.125152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcb12a-5012-4df0-b18e-ce98525e6bc0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.125450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcb12a-5012-4df0-b18e-ce98525e6bc0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.125717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcb12a-5012-4df0-b18e-ce98525e6bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.131387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:53.143842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c7bcb12a-5012-4df0-b18e-ce98525e6bc0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:53.145066 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c7bcb12a-5012-4df0-b18e-ce98525e6bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:53.185176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea02244-4b20-45c0-9880-fcf8d24194e5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:53.220953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 222/445] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:37:53.235186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c423ae-a724-466b-9ced-bc7f0596cf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:53.235976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c423ae-a724-466b-9ced-bc7f0596cf22 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.236331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c423ae-a724-466b-9ced-bc7f0596cf22 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.236741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c423ae-a724-466b-9ced-bc7f0596cf22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.247086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcb12a-5012-4df0-b18e-ce98525e6bc0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:53.285117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:53.316526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c423ae-a724-466b-9ced-bc7f0596cf22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:53.318494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c423ae-a724-466b-9ced-bc7f0596cf22 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:53.358880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcb12a-5012-4df0-b18e-ce98525e6bc0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:53.385774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:53.405448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6683-bf60-4f2c-8256-ed59be907f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:53.406384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6683-bf60-4f2c-8256-ed59be907f48 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.406706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6683-bf60-4f2c-8256-ed59be907f48 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.406989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b6683-bf60-4f2c-8256-ed59be907f48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.419100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b6683-bf60-4f2c-8256-ed59be907f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-7e3b6683-bf60-4f2c-8256-ed59be907f48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:53.427503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:53.428384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:53.429970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 223/446] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:53.435080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6683-bf60-4f2c-8256-ed59be907f48 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:53.447627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 224/447] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:37:53.453309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c423ae-a724-466b-9ced-bc7f0596cf22 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:53.547570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:53.553479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b6683-bf60-4f2c-8256-ed59be907f48 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:53.565875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c423ae-a724-466b-9ced-bc7f0596cf22 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:53.587756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e5c0e-d4e5-47ae-97ef-ccc4fd9b08f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:53.588419 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e5c0e-d4e5-47ae-97ef-ccc4fd9b08f5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.588709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e5c0e-d4e5-47ae-97ef-ccc4fd9b08f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.588993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e5c0e-d4e5-47ae-97ef-ccc4fd9b08f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.589731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e5c0e-d4e5-47ae-97ef-ccc4fd9b08f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:53.590076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e5c0e-d4e5-47ae-97ef-ccc4fd9b08f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:53.591451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e5c0e-d4e5-47ae-97ef-ccc4fd9b08f5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:53.599082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1342c13-556b-43ae-9f0b-b5a3ae1414a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:53.599556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 224/448] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:53.600011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1342c13-556b-43ae-9f0b-b5a3ae1414a1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.600355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1342c13-556b-43ae-9f0b-b5a3ae1414a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.600631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1342c13-556b-43ae-9f0b-b5a3ae1414a1 None admin] PATH_INFO: `/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.602498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1342c13-556b-43ae-9f0b-b5a3ae1414a1 None admin] RBAC: Authorizing `identity:get_project(project_id=53d2a61c9fa84503ab8a6c4eb2d3f27e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:53.603924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1342c13-556b-43ae-9f0b-b5a3ae1414a1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:53.605244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e5c0e-d4e5-47ae-97ef-ccc4fd9b08f5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:53.605665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 225/449] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:53 2022] GET /identity/v3/projects/53d2a61c9fa84503ab8a6c4eb2d3f27e => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:53.610915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1342c13-556b-43ae-9f0b-b5a3ae1414a1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:53.631539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:53.703213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e5c0e-d4e5-47ae-97ef-ccc4fd9b08f5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:53.706522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1342c13-556b-43ae-9f0b-b5a3ae1414a1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:53.738725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4464b3-df97-4bb8-bcba-45178683348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:53.739369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4464b3-df97-4bb8-bcba-45178683348f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.739521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8d98d-89c4-4a4b-98ee-8ff9aea417a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:53.739645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4464b3-df97-4bb8-bcba-45178683348f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.739918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4464b3-df97-4bb8-bcba-45178683348f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.740366 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8d98d-89c4-4a4b-98ee-8ff9aea417a6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.740647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8d98d-89c4-4a4b-98ee-8ff9aea417a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.740921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8d98d-89c4-4a4b-98ee-8ff9aea417a6 None admin] PATH_INFO: `/v3/groups/51186006fd854cbc846ca13ba62d9289` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.742246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8d98d-89c4-4a4b-98ee-8ff9aea417a6 None admin] RBAC: Authorizing `identity:get_group(group_id=51186006fd854cbc846ca13ba62d9289)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:53.743680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8d98d-89c4-4a4b-98ee-8ff9aea417a6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:53.745265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 226/450] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:53 2022] GET /identity/v3/groups/51186006fd854cbc846ca13ba62d9289 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:53.751202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8d98d-89c4-4a4b-98ee-8ff9aea417a6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:53.767132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:53.807876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4464b3-df97-4bb8-bcba-45178683348f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:53.809311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4464b3-df97-4bb8-bcba-45178683348f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:53.837981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:53.849484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8d98d-89c4-4a4b-98ee-8ff9aea417a6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:53.881575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308090d-5238-405c-9c5e-5a5c7ed72f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:53.882762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308090d-5238-405c-9c5e-5a5c7ed72f3d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:53.882986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308090d-5238-405c-9c5e-5a5c7ed72f3d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:53.883431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308090d-5238-405c-9c5e-5a5c7ed72f3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:53.884508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308090d-5238-405c-9c5e-5a5c7ed72f3d None admin] RBAC: Adding query filter params (group.id=51186006fd854cbc846ca13ba62d9289, role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=53d2a61c9fa84503ab8a6c4eb2d3f27e) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:53.884815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308090d-5238-405c-9c5e-5a5c7ed72f3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:53.886271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308090d-5238-405c-9c5e-5a5c7ed72f3d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:53.892056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 227/451] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:53 2022] GET /identity/v3/role_assignments?group.id=51186006fd854cbc846ca13ba62d9289&scope.project.id=53d2a61c9fa84503ab8a6c4eb2d3f27e&role.id=fe4d636f5c4842f18569347911e6cdec => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:53.906743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 225/452] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 09:37:53.912615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308090d-5238-405c-9c5e-5a5c7ed72f3d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:54.015715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308090d-5238-405c-9c5e-5a5c7ed72f3d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:54.051463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d9d07-1e39-4dc3-9b6b-af59aab8e82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:54.052337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d9d07-1e39-4dc3-9b6b-af59aab8e82c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:54.052622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d9d07-1e39-4dc3-9b6b-af59aab8e82c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:54.052901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d9d07-1e39-4dc3-9b6b-af59aab8e82c None admin] PATH_INFO: `/v3/projects/a91190fa89bd41f49048e6128973a986` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:54.054743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d9d07-1e39-4dc3-9b6b-af59aab8e82c None admin] RBAC: Authorizing `identity:get_project(project_id=a91190fa89bd41f49048e6128973a986)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:54.056208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d9d07-1e39-4dc3-9b6b-af59aab8e82c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:54.058007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 228/453] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:37:53 2022] GET /identity/v3/projects/a91190fa89bd41f49048e6128973a986 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:54.063617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4464b3-df97-4bb8-bcba-45178683348f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:54.123509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:54.166942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4464b3-df97-4bb8-bcba-45178683348f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:54.197255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb89e6-5a5a-4acb-83ca-20fef626c929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:54.198127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb89e6-5a5a-4acb-83ca-20fef626c929 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:54.198406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb89e6-5a5a-4acb-83ca-20fef626c929 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:54.198674 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb89e6-5a5a-4acb-83ca-20fef626c929 None admin] PATH_INFO: `/v3/groups/604ee7930c4b48c486e6532caf56a94e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:54.199807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb89e6-5a5a-4acb-83ca-20fef626c929 None admin] RBAC: Authorizing `identity:get_group(group_id=604ee7930c4b48c486e6532caf56a94e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:54.201226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb89e6-5a5a-4acb-83ca-20fef626c929 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:54.202826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 226/454] 10.222.0.42 () {56 vars in 1180 bytes} [Fri Jun 3 09:37:54 2022] GET /identity/v3/groups/604ee7930c4b48c486e6532caf56a94e => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:54.208621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d9d07-1e39-4dc3-9b6b-af59aab8e82c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:54.230890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:54.305334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d9d07-1e39-4dc3-9b6b-af59aab8e82c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:54.337297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59361f8b-3be7-4d82-8d26-7e8fa3474637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:54.337960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59361f8b-3be7-4d82-8d26-7e8fa3474637 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:54.338246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59361f8b-3be7-4d82-8d26-7e8fa3474637 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:54.338524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59361f8b-3be7-4d82-8d26-7e8fa3474637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:54.339342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59361f8b-3be7-4d82-8d26-7e8fa3474637 None admin] RBAC: Adding query filter params (group.id=604ee7930c4b48c486e6532caf56a94e, role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=a91190fa89bd41f49048e6128973a986) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:54.339652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59361f8b-3be7-4d82-8d26-7e8fa3474637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:54.341082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59361f8b-3be7-4d82-8d26-7e8fa3474637 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:54.346800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 229/455] 10.222.0.42 () {56 vars in 1376 bytes} [Fri Jun 3 09:37:54 2022] GET /identity/v3/role_assignments?group.id=604ee7930c4b48c486e6532caf56a94e&scope.project.id=a91190fa89bd41f49048e6128973a986&role.id=7583ad887d8b4052bd11783371c338b4 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:57.173629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8607f2-e822-4d89-9e53-41d704ff2091 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.174190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8607f2-e822-4d89-9e53-41d704ff2091 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.174190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8607f2-e822-4d89-9e53-41d704ff2091 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.175794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 227/456] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:57.181800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa745d69-bad6-4127-a1e8-373ae3009806 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.182658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa745d69-bad6-4127-a1e8-373ae3009806 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.183157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa745d69-bad6-4127-a1e8-373ae3009806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.203620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-aa745d69-bad6-4127-a1e8-373ae3009806 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:57.205111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-aa745d69-bad6-4127-a1e8-373ae3009806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:57.296551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 230/457] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:57.327973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8607f2-e822-4d89-9e53-41d704ff2091 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:57.424881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127f97c-732e-4e53-b92b-8c1645d68ea0 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.425575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127f97c-732e-4e53-b92b-8c1645d68ea0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.426072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127f97c-732e-4e53-b92b-8c1645d68ea0 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.428942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 231/458] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:57.434259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be948444-7ad2-49ba-8b46-3e59e8bfd87a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.434811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be948444-7ad2-49ba-8b46-3e59e8bfd87a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.435289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be948444-7ad2-49ba-8b46-3e59e8bfd87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.448964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8607f2-e822-4d89-9e53-41d704ff2091 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:57.474459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-be948444-7ad2-49ba-8b46-3e59e8bfd87a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:57.476317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-be948444-7ad2-49ba-8b46-3e59e8bfd87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:57.495947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:57.506357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:57.521979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5401f80-05d9-4c3b-884f-4c009d0adbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:57.521979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5401f80-05d9-4c3b-884f-4c009d0adbaa None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.521979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5401f80-05d9-4c3b-884f-4c009d0adbaa None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.521979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5401f80-05d9-4c3b-884f-4c009d0adbaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.528278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5401f80-05d9-4c3b-884f-4c009d0adbaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:57.529731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5401f80-05d9-4c3b-884f-4c009d0adbaa None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d5401f80-05d9-4c3b-884f-4c009d0adbaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:37:57.548345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:57.549548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:57.550818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:37:57.550818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:37:57.550818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:37:57.550818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:57.550818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 228/459] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:57.555635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5401f80-05d9-4c3b-884f-4c009d0adbaa None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:57.607493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 232/460] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:57.649514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be948444-7ad2-49ba-8b46-3e59e8bfd87a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:57.701484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5401f80-05d9-4c3b-884f-4c009d0adbaa None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:57.749770 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7de585-2701-4c5a-9a02-b81e86581f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:57.750747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7de585-2701-4c5a-9a02-b81e86581f37 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.751244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7de585-2701-4c5a-9a02-b81e86581f37 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.751696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7de585-2701-4c5a-9a02-b81e86581f37 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.752723 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7de585-2701-4c5a-9a02-b81e86581f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:57.753215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7de585-2701-4c5a-9a02-b81e86581f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:57.754785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7de585-2701-4c5a-9a02-b81e86581f37 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:57.765427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be948444-7ad2-49ba-8b46-3e59e8bfd87a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:57.769120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 229/461] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:37:57.770767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358856c-6c8b-480c-abb6-b3f1581f8c2a None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.771233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358856c-6c8b-480c-abb6-b3f1581f8c2a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.771682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358856c-6c8b-480c-abb6-b3f1581f8c2a None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.778584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 230/462] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:57.779231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358856c-6c8b-480c-abb6-b3f1581f8c2a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:57.802949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d2565-ffc2-4013-b756-764426149ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:57.803746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2565-ffc2-4013-b756-764426149ce4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.804692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2565-ffc2-4013-b756-764426149ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.804974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2565-ffc2-4013-b756-764426149ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.815687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d2565-ffc2-4013-b756-764426149ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:57.817177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d2565-ffc2-4013-b756-764426149ce4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:57.819902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:57.827923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-454d2565-ffc2-4013-b756-764426149ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:37:57.837990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:57.839075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:57.840024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:37:57.840024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:37:57.840024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:37:57.840024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:57.840024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 233/463] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:57.840355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5379d0d-b315-4fc8-8997-c84cd3e9eb4d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.840744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5379d0d-b315-4fc8-8997-c84cd3e9eb4d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.842218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5379d0d-b315-4fc8-8997-c84cd3e9eb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.858518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d5379d0d-b315-4fc8-8997-c84cd3e9eb4d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:57.859779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d5379d0d-b315-4fc8-8997-c84cd3e9eb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:57.895379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358856c-6c8b-480c-abb6-b3f1581f8c2a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:57.930738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6181e8-00d6-4041-aa40-a2e9fda2fe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:57.931408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6181e8-00d6-4041-aa40-a2e9fda2fe4f None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.931732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6181e8-00d6-4041-aa40-a2e9fda2fe4f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.932008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6181e8-00d6-4041-aa40-a2e9fda2fe4f None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.932662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6181e8-00d6-4041-aa40-a2e9fda2fe4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:57.934123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6181e8-00d6-4041-aa40-a2e9fda2fe4f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:57.935282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 234/464] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:57.936590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4f7f2-1ae2-4265-b0ea-b5f561db3485 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:57.936879 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4f7f2-1ae2-4265-b0ea-b5f561db3485 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:57.937158 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4f7f2-1ae2-4265-b0ea-b5f561db3485 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:57.941944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 235/465] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:57.941944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4f7f2-1ae2-4265-b0ea-b5f561db3485 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:58.031978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 231/466] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:57 2022] POST /identity/v3/users => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:58.033509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3bfca-8d7b-4df3-9dcf-b7e113a0e5a9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:58.034020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3bfca-8d7b-4df3-9dcf-b7e113a0e5a9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:58.034479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3bfca-8d7b-4df3-9dcf-b7e113a0e5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:58.060195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4f7f2-1ae2-4265-b0ea-b5f561db3485 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:58.060563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0cd3bfca-8d7b-4df3-9dcf-b7e113a0e5a9 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:58.062269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0cd3bfca-8d7b-4df3-9dcf-b7e113a0e5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:58.102288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:58.115028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a4443-4fa8-4870-986b-66f9234e36ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:58.115720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a4443-4fa8-4870-986b-66f9234e36ed None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:58.116016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a4443-4fa8-4870-986b-66f9234e36ed None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:58.116814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a4443-4fa8-4870-986b-66f9234e36ed None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:58.127781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a4443-4fa8-4870-986b-66f9234e36ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:58.127781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a4443-4fa8-4870-986b-66f9234e36ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:58.127781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a4443-4fa8-4870-986b-66f9234e36ed None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:58.146294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 236/467] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:58.151017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a4443-4fa8-4870-986b-66f9234e36ed None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:58.159196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 232/468] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:58.160359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3bfca-8d7b-4df3-9dcf-b7e113a0e5a9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:58.195084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:58.283307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3bfca-8d7b-4df3-9dcf-b7e113a0e5a9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:58.323106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083610dd-9ef7-49b2-80b9-e8a6bdbd0dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:58.323953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083610dd-9ef7-49b2-80b9-e8a6bdbd0dca None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:58.324976 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083610dd-9ef7-49b2-80b9-e8a6bdbd0dca None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:58.324976 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083610dd-9ef7-49b2-80b9-e8a6bdbd0dca None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:58.325296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083610dd-9ef7-49b2-80b9-e8a6bdbd0dca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:58.328124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083610dd-9ef7-49b2-80b9-e8a6bdbd0dca None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:58.332579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a4443-4fa8-4870-986b-66f9234e36ed None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:58.380943 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b1f9e-225b-4ea3-884c-ee991b8aef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:58.382634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b1f9e-225b-4ea3-884c-ee991b8aef2e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:58.383093 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b1f9e-225b-4ea3-884c-ee991b8aef2e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:58.383561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b1f9e-225b-4ea3-884c-ee991b8aef2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:58.395881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b1f9e-225b-4ea3-884c-ee991b8aef2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:58.397786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b1f9e-225b-4ea3-884c-ee991b8aef2e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-817b1f9e-225b-4ea3-884c-ee991b8aef2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:37:58.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:58.426116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:58.426970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:37:58.426970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:37:58.426970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:37:58.426970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:58.427614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 237/469] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:58.428613 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b1f9e-225b-4ea3-884c-ee991b8aef2e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:58.468775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 233/470] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:58 2022] POST /identity/v3/users => generated 248 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:58.469655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083610dd-9ef7-49b2-80b9-e8a6bdbd0dca None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:58.474284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:58.547776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b1f9e-225b-4ea3-884c-ee991b8aef2e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:58.616044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27316e-fa11-4b1f-9ed2-f0239c4ead03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:58.617855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27316e-fa11-4b1f-9ed2-f0239c4ead03 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:58.618394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27316e-fa11-4b1f-9ed2-f0239c4ead03 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:58.618829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27316e-fa11-4b1f-9ed2-f0239c4ead03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:58.640995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27316e-fa11-4b1f-9ed2-f0239c4ead03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:58.649923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27316e-fa11-4b1f-9ed2-f0239c4ead03 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-bf27316e-fa11-4b1f-9ed2-f0239c4ead03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:37:58.703629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:58.704513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:58.705325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:37:58.705325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:37:58.705325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:37:58.705325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:58.719992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 238/471] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:58.721603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:58.724453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27316e-fa11-4b1f-9ed2-f0239c4ead03 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:58.766443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083610dd-9ef7-49b2-80b9-e8a6bdbd0dca None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:58.835871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8e89c-81e4-427b-9a0d-ff02efae8d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:58.836913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8e89c-81e4-427b-9a0d-ff02efae8d9e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:58.837392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8e89c-81e4-427b-9a0d-ff02efae8d9e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:58.837876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8e89c-81e4-427b-9a0d-ff02efae8d9e None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:58.838642 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8e89c-81e4-427b-9a0d-ff02efae8d9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:58.838948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8e89c-81e4-427b-9a0d-ff02efae8d9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:58.840317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8e89c-81e4-427b-9a0d-ff02efae8d9e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:58.852737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 234/472] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:58.854002 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e8847-36a3-48f0-be21-2aae7b279eb4 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:58.854292 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e8847-36a3-48f0-be21-2aae7b279eb4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:58.854570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e8847-36a3-48f0-be21-2aae7b279eb4 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:58.855913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 235/473] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:37:58.862515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:58.864991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e8847-36a3-48f0-be21-2aae7b279eb4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:58.868561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27316e-fa11-4b1f-9ed2-f0239c4ead03 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:58.982533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e8847-36a3-48f0-be21-2aae7b279eb4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:59.021610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef98c7-f56f-4843-ac1c-fba1d33ff607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:59.022748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef98c7-f56f-4843-ac1c-fba1d33ff607 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:59.023260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef98c7-f56f-4843-ac1c-fba1d33ff607 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:59.023741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef98c7-f56f-4843-ac1c-fba1d33ff607 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:59.024992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef98c7-f56f-4843-ac1c-fba1d33ff607 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:59.026758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef98c7-f56f-4843-ac1c-fba1d33ff607 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:59.039182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:59.071671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261446b-8a4c-4ba8-bcc1-758eed71e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:59.072334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261446b-8a4c-4ba8-bcc1-758eed71e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:59.072615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261446b-8a4c-4ba8-bcc1-758eed71e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:59.073010 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261446b-8a4c-4ba8-bcc1-758eed71e5b8 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:59.073734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261446b-8a4c-4ba8-bcc1-758eed71e5b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:59.074080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261446b-8a4c-4ba8-bcc1-758eed71e5b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:59.075446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261446b-8a4c-4ba8-bcc1-758eed71e5b8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:59.087661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 239/474] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:37:59.092517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3e6a8-256a-49d4-afd9-b72a64f1e0c6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:59.092517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3e6a8-256a-49d4-afd9-b72a64f1e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:59.092517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3e6a8-256a-49d4-afd9-b72a64f1e0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:59.107478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-56e3e6a8-256a-49d4-afd9-b72a64f1e0c6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:37:59.108648 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-56e3e6a8-256a-49d4-afd9-b72a64f1e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:37:59.176807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 240/475] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:37:59.181919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3e6a8-256a-49d4-afd9-b72a64f1e0c6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:59.201427 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 236/476] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:58 2022] POST /identity/v3/users => generated 248 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:37:59.213875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef98c7-f56f-4843-ac1c-fba1d33ff607 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:59.331287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:59.354975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3e6a8-256a-49d4-afd9-b72a64f1e0c6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:59.409934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769e738-f6fe-4f5e-9b76-0e87e869b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:59.411003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e738-f6fe-4f5e-9b76-0e87e869b02b None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:59.411586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e738-f6fe-4f5e-9b76-0e87e869b02b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:59.412085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769e738-f6fe-4f5e-9b76-0e87e869b02b None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:59.413019 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769e738-f6fe-4f5e-9b76-0e87e869b02b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:59.414910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769e738-f6fe-4f5e-9b76-0e87e869b02b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:59.435333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef98c7-f56f-4843-ac1c-fba1d33ff607 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:59.476331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:59.477418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:59.477929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:59.478408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:59.479350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:59.480951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:59.511830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 241/477] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:37:59 2022] POST /identity/v3/users => generated 249 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:37:59.528115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.sql.core [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 03 09:37:59.528115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 03 09:37:59.528115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [parameters: {'id': '2085c5a2adf94f2c87de58708e05750f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 03 09:37:59.528115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87026) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application conn.query(q) Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application result.read() Jun 03 09:37:59.546507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return ref.to_dict() Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 09:37:59.547426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application next(self.gen) Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application yield resource Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application next(self.gen) Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.session.rollback() Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.force_reraise() Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise self.value Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application session.commit() Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 03 09:37:59.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._prepare_impl() Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.session.flush() Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._flush(objects) Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application compat.raise_( Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application flush_context.execute() Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rec.execute(self) Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 03 09:37:59.549213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 09:37:59.550099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application conn.query(q) Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application result.read() Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 09:37:59.550891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [parameters: {'id': '2085c5a2adf94f2c87de58708e05750f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:59.551656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 03 09:37:59.552421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:37:59.556589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 237/478] 10.222.0.42 () {60 vars in 1131 bytes} [Fri Jun 3 09:37:59 2022] POST /identity/v3/roles => generated 144 bytes in 343 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:37:59.562959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769e738-f6fe-4f5e-9b76-0e87e869b02b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:59.679079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769e738-f6fe-4f5e-9b76-0e87e869b02b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:59.733246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38609f2a-3de3-457c-bf94-39508cb20443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:59.734504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38609f2a-3de3-457c-bf94-39508cb20443 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:59.735014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38609f2a-3de3-457c-bf94-39508cb20443 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:59.735492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38609f2a-3de3-457c-bf94-39508cb20443 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:59.743129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:59.754689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38609f2a-3de3-457c-bf94-39508cb20443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-38609f2a-3de3-457c-bf94-39508cb20443 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:37:59.763556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:37:59.764397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:37:59.765164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 242/479] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:37:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:37:59.770725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:37:59.838064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:37:59.883704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad592cb7-6399-491c-b220-defaf1c31395 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:37:59.920613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd3e05-d63e-476a-8f7a-7e58b13fa3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:37:59.921287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3e05-d63e-476a-8f7a-7e58b13fa3ba None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:37:59.921571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3e05-d63e-476a-8f7a-7e58b13fa3ba None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:37:59.921850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3e05-d63e-476a-8f7a-7e58b13fa3ba None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:37:59.922646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd3e05-d63e-476a-8f7a-7e58b13fa3ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:37:59.922952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd3e05-d63e-476a-8f7a-7e58b13fa3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:37:59.924326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd3e05-d63e-476a-8f7a-7e58b13fa3ba None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:37:59.934078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 238/480] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:37:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:00.582875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5da2d-322a-456a-b04b-9cf32c2acdca None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:00.583412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5da2d-322a-456a-b04b-9cf32c2acdca None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:00.583412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5da2d-322a-456a-b04b-9cf32c2acdca None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:00.585652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 243/481] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:00.591255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c629f-a5d6-4637-8730-1dec8e55cac4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:00.591716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c629f-a5d6-4637-8730-1dec8e55cac4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:00.591803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c629f-a5d6-4637-8730-1dec8e55cac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:00.614873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-664c629f-a5d6-4637-8730-1dec8e55cac4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:00.616076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-664c629f-a5d6-4637-8730-1dec8e55cac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:00.706619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 239/482] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:00.739520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5da2d-322a-456a-b04b-9cf32c2acdca None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:00.882673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5da2d-322a-456a-b04b-9cf32c2acdca None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:00.924804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4f60c-861b-4fbb-9adb-9d2e9b33a8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:00.925992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4f60c-861b-4fbb-9adb-9d2e9b33a8db None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:00.926468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4f60c-861b-4fbb-9adb-9d2e9b33a8db None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:00.926953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4f60c-861b-4fbb-9adb-9d2e9b33a8db None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:00.940316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4f60c-861b-4fbb-9adb-9d2e9b33a8db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3dc4f60c-861b-4fbb-9adb-9d2e9b33a8db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:00.948214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:00.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:00.949819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:00.956540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 244/483] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:00.962673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c629f-a5d6-4637-8730-1dec8e55cac4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:01.043026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:01.107129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c629f-a5d6-4637-8730-1dec8e55cac4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:01.149820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08b1ce-137f-45a3-8c8c-efa69e195403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:01.150461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08b1ce-137f-45a3-8c8c-efa69e195403 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:01.150742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08b1ce-137f-45a3-8c8c-efa69e195403 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:01.151015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08b1ce-137f-45a3-8c8c-efa69e195403 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:01.151759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08b1ce-137f-45a3-8c8c-efa69e195403 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:01.152061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08b1ce-137f-45a3-8c8c-efa69e195403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:01.153423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08b1ce-137f-45a3-8c8c-efa69e195403 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:01.162110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 240/484] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:01.167731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4f60c-861b-4fbb-9adb-9d2e9b33a8db None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:01.237484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:01.295737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4f60c-861b-4fbb-9adb-9d2e9b33a8db None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:01.331694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc43b15-0fb0-4bae-bd38-9cd5e7e4b4f6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:01.331992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc43b15-0fb0-4bae-bd38-9cd5e7e4b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:01.332734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc43b15-0fb0-4bae-bd38-9cd5e7e4b4f6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:01.334804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 241/485] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:01.341912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de8ac0-74d8-48ae-9643-118cbed5034b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:01.342197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de8ac0-74d8-48ae-9643-118cbed5034b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:01.342798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de8ac0-74d8-48ae-9643-118cbed5034b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:01.352387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1404-a059-4806-bda9-3dc0411f9288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:01.353188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1404-a059-4806-bda9-3dc0411f9288 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:01.353472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1404-a059-4806-bda9-3dc0411f9288 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:01.353749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f1404-a059-4806-bda9-3dc0411f9288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:01.375148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-10de8ac0-74d8-48ae-9643-118cbed5034b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:01.376349 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-10de8ac0-74d8-48ae-9643-118cbed5034b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:01.379044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f1404-a059-4806-bda9-3dc0411f9288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:01.380569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f1404-a059-4806-bda9-3dc0411f9288 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-6a3f1404-a059-4806-bda9-3dc0411f9288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:01.405678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:01.406550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:01.407383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:01.407383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:01.407383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:01.407383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:01.408119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 245/486] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:01.417327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1404-a059-4806-bda9-3dc0411f9288 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:01.494661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 242/487] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:01.518623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de8ac0-74d8-48ae-9643-118cbed5034b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:01.588042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:01.648657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f1404-a059-4806-bda9-3dc0411f9288 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:01.658000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:01.688034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4835bc-68e8-4a6e-8a5f-a5b8acd3b74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:01.688705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4835bc-68e8-4a6e-8a5f-a5b8acd3b74e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:01.688984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4835bc-68e8-4a6e-8a5f-a5b8acd3b74e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:01.689256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4835bc-68e8-4a6e-8a5f-a5b8acd3b74e None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:01.689989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4835bc-68e8-4a6e-8a5f-a5b8acd3b74e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:01.690294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4835bc-68e8-4a6e-8a5f-a5b8acd3b74e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:01.691662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4835bc-68e8-4a6e-8a5f-a5b8acd3b74e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:01.695667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de8ac0-74d8-48ae-9643-118cbed5034b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:01.715396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 246/488] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:01.722640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4835bc-68e8-4a6e-8a5f-a5b8acd3b74e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:01.774689 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d9dbc-0957-4ab0-bb46-89559904f2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:01.775511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d9dbc-0957-4ab0-bb46-89559904f2a3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:01.775796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d9dbc-0957-4ab0-bb46-89559904f2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:01.776074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d9dbc-0957-4ab0-bb46-89559904f2a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:01.799113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d9dbc-0957-4ab0-bb46-89559904f2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-4e0d9dbc-0957-4ab0-bb46-89559904f2a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:01.815978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:01.816906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:01.818170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 243/489] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:01.823389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d9dbc-0957-4ab0-bb46-89559904f2a3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:01.835582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4835bc-68e8-4a6e-8a5f-a5b8acd3b74e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:01.878935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7476f3d9-c6d5-4961-8982-0a081abee78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:01.880486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476f3d9-c6d5-4961-8982-0a081abee78c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:01.880780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476f3d9-c6d5-4961-8982-0a081abee78c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:01.881055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7476f3d9-c6d5-4961-8982-0a081abee78c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:01.898515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7476f3d9-c6d5-4961-8982-0a081abee78c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:01.899981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7476f3d9-c6d5-4961-8982-0a081abee78c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7476f3d9-c6d5-4961-8982-0a081abee78c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:01.926447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:01.927350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:01.928866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 247/490] 10.222.0.42 () {56 vars in 1093 bytes} [Fri Jun 3 09:38:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:01.929436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e252-21fb-4eb8-b780-6005ebb2c2e0 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:01.929436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e252-21fb-4eb8-b780-6005ebb2c2e0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:01.929769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e252-21fb-4eb8-b780-6005ebb2c2e0 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:01.931200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 248/491] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:01.933083 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e252-21fb-4eb8-b780-6005ebb2c2e0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:01.974286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:02.001823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d9dbc-0957-4ab0-bb46-89559904f2a3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:02.006016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:02.039487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7509cdf-e3e3-4256-bbf5-59aa41ea7e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:02.040414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7509cdf-e3e3-4256-bbf5-59aa41ea7e45 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.040696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7509cdf-e3e3-4256-bbf5-59aa41ea7e45 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.040970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7509cdf-e3e3-4256-bbf5-59aa41ea7e45 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.041710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7509cdf-e3e3-4256-bbf5-59aa41ea7e45 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:02.042043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7509cdf-e3e3-4256-bbf5-59aa41ea7e45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:02.043420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7509cdf-e3e3-4256-bbf5-59aa41ea7e45 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:02.052438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 244/492] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:02.054373 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0cfb9-ad05-4c8d-bdcd-b53b3e2faf43 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.054658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0cfb9-ad05-4c8d-bdcd-b53b3e2faf43 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.054934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0cfb9-ad05-4c8d-bdcd-b53b3e2faf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.063434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e252-21fb-4eb8-b780-6005ebb2c2e0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:02.071617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-61c0cfb9-ad05-4c8d-bdcd-b53b3e2faf43 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:02.072915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-61c0cfb9-ad05-4c8d-bdcd-b53b3e2faf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:02.106603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62301d46-8ac4-4e10-b40c-e2d72841b349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:02.107627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62301d46-8ac4-4e10-b40c-e2d72841b349 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.108401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62301d46-8ac4-4e10-b40c-e2d72841b349 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.109784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62301d46-8ac4-4e10-b40c-e2d72841b349 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.110868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62301d46-8ac4-4e10-b40c-e2d72841b349 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:02.111400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62301d46-8ac4-4e10-b40c-e2d72841b349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:02.113075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62301d46-8ac4-4e10-b40c-e2d72841b349 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:02.149907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 245/493] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:02.149907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0cfb9-ad05-4c8d-bdcd-b53b3e2faf43 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:02.175917 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 249/494] 10.222.0.42 () {56 vars in 1133 bytes} [Fri Jun 3 09:38:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:02.176685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62301d46-8ac4-4e10-b40c-e2d72841b349 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:02.274311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:02.281119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0cfb9-ad05-4c8d-bdcd-b53b3e2faf43 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:02.281359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62301d46-8ac4-4e10-b40c-e2d72841b349 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:02.290981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:02.315863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1a63d-c264-4b26-aa5c-e0a0a936b6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:02.316174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59c070-c612-408f-926f-db1119fe63ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:02.316657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1a63d-c264-4b26-aa5c-e0a0a936b6e3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.316927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59c070-c612-408f-926f-db1119fe63ab None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.317022 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1a63d-c264-4b26-aa5c-e0a0a936b6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.317206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59c070-c612-408f-926f-db1119fe63ab None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.317299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1a63d-c264-4b26-aa5c-e0a0a936b6e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.317477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59c070-c612-408f-926f-db1119fe63ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.328068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59c070-c612-408f-926f-db1119fe63ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:02.328579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1a63d-c264-4b26-aa5c-e0a0a936b6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:02.330073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1a63d-c264-4b26-aa5c-e0a0a936b6e3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8a59c070-c612-408f-926f-db1119fe63ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:02.336377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:02.337234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:02.338055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 250/495] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:02.339370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59c070-c612-408f-926f-db1119fe63ab None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-bde1a63d-c264-4b26-aa5c-e0a0a936b6e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:02.354461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:02.355311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:02.356132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:02.356132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:02.356132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:02.356132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:02.357693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 246/496] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:02.359131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395048e-49d7-412f-8747-f0736f7d9249 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.359558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395048e-49d7-412f-8747-f0736f7d9249 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.359835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395048e-49d7-412f-8747-f0736f7d9249 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.361225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 247/497] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:02.362623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395048e-49d7-412f-8747-f0736f7d9249 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:02.446507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59c070-c612-408f-926f-db1119fe63ab None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:02.472925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395048e-49d7-412f-8747-f0736f7d9249 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:02.482125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acd7e-7ada-4621-9aed-d64d7b373b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:02.482780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acd7e-7ada-4621-9aed-d64d7b373b67 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.483347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acd7e-7ada-4621-9aed-d64d7b373b67 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.483347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acd7e-7ada-4621-9aed-d64d7b373b67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.498914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:02.514444 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce113-9bb4-4dd1-8a3d-b0f032ef812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:02.515219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce113-9bb4-4dd1-8a3d-b0f032ef812b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.515497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce113-9bb4-4dd1-8a3d-b0f032ef812b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.515819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce113-9bb4-4dd1-8a3d-b0f032ef812b None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.516708 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce113-9bb4-4dd1-8a3d-b0f032ef812b None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:02.517012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce113-9bb4-4dd1-8a3d-b0f032ef812b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:02.519141 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce113-9bb4-4dd1-8a3d-b0f032ef812b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:02.530358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 248/498] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:02.530975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce113-9bb4-4dd1-8a3d-b0f032ef812b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:02.544134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:02.558301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acd7e-7ada-4621-9aed-d64d7b373b67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:02.559937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acd7e-7ada-4621-9aed-d64d7b373b67 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:02.631168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce113-9bb4-4dd1-8a3d-b0f032ef812b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:02.661817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 251/499] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:02.664360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3771966-e2c8-487a-a35a-4491f3a73f65 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.664644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3771966-e2c8-487a-a35a-4491f3a73f65 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.664924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3771966-e2c8-487a-a35a-4491f3a73f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.669821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0dfd9-1f2c-45f4-9088-f6d6da996bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:02.670506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0dfd9-1f2c-45f4-9088-f6d6da996bdc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.670790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0dfd9-1f2c-45f4-9088-f6d6da996bdc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.671076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0dfd9-1f2c-45f4-9088-f6d6da996bdc None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.671783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0dfd9-1f2c-45f4-9088-f6d6da996bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:02.672088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0dfd9-1f2c-45f4-9088-f6d6da996bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:02.673453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0dfd9-1f2c-45f4-9088-f6d6da996bdc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:02.682852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a3771966-e2c8-487a-a35a-4491f3a73f65 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:02.684034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a3771966-e2c8-487a-a35a-4491f3a73f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:02.689871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 249/500] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:02.689871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0dfd9-1f2c-45f4-9088-f6d6da996bdc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:02.743939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:02.766955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:02.768385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 252/501] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:02.769304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3771966-e2c8-487a-a35a-4491f3a73f65 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:02.813470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0dfd9-1f2c-45f4-9088-f6d6da996bdc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:02.857030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93ddbb-0287-4c30-9f57-e786fd795b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:02.857822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93ddbb-0287-4c30-9f57-e786fd795b83 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.858132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93ddbb-0287-4c30-9f57-e786fd795b83 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.858406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93ddbb-0287-4c30-9f57-e786fd795b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.869169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93ddbb-0287-4c30-9f57-e786fd795b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:02.870635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93ddbb-0287-4c30-9f57-e786fd795b83 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:02.881864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3771966-e2c8-487a-a35a-4491f3a73f65 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-da93ddbb-0287-4c30-9f57-e786fd795b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:02.889595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:02.890451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:02.896190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:02.896190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:02.896190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:02.896190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:02.896190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 250/502] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:02.896190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93ddbb-0287-4c30-9f57-e786fd795b83 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:02.918056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa862b86-683c-4d8e-9a12-bb319f7bc042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:02.918844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa862b86-683c-4d8e-9a12-bb319f7bc042 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:02.919125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa862b86-683c-4d8e-9a12-bb319f7bc042 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:02.919398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa862b86-683c-4d8e-9a12-bb319f7bc042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:02.929997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa862b86-683c-4d8e-9a12-bb319f7bc042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:02.931452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa862b86-683c-4d8e-9a12-bb319f7bc042 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-aa862b86-683c-4d8e-9a12-bb319f7bc042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:02.951142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:02.952099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:02.953124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:02.953124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:02.953124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:02.953124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:02.957014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 253/503] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:02.957833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa862b86-683c-4d8e-9a12-bb319f7bc042 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:02.988889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.001203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93ddbb-0287-4c30-9f57-e786fd795b83 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.025924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.037058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f7cec-a337-49ae-8733-809a46146514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.037855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f7cec-a337-49ae-8733-809a46146514 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.038190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f7cec-a337-49ae-8733-809a46146514 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.038463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f7cec-a337-49ae-8733-809a46146514 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.045802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f7cec-a337-49ae-8733-809a46146514 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.047268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f7cec-a337-49ae-8733-809a46146514 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.061265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa862b86-683c-4d8e-9a12-bb319f7bc042 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-483f7cec-a337-49ae-8733-809a46146514 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:03.064847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:03.065740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 251/504] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:03.067214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f7cec-a337-49ae-8733-809a46146514 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.102532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35a9db-2d58-4fe7-b8ae-8a62d0f9f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.103328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35a9db-2d58-4fe7-b8ae-8a62d0f9f37b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.103614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35a9db-2d58-4fe7-b8ae-8a62d0f9f37b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.103888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35a9db-2d58-4fe7-b8ae-8a62d0f9f37b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.115160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35a9db-2d58-4fe7-b8ae-8a62d0f9f37b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-dd35a9db-2d58-4fe7-b8ae-8a62d0f9f37b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:03.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:03.124453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:03.125203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 254/505] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:03.125860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35a9db-2d58-4fe7-b8ae-8a62d0f9f37b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.174301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f7cec-a337-49ae-8733-809a46146514 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.207464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e6102-cb9e-4834-bd4e-23562f88436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.208118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e6102-cb9e-4834-bd4e-23562f88436d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.208401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e6102-cb9e-4834-bd4e-23562f88436d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.208680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e6102-cb9e-4834-bd4e-23562f88436d None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.209398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e6102-cb9e-4834-bd4e-23562f88436d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:03.209704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e6102-cb9e-4834-bd4e-23562f88436d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.211626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e6102-cb9e-4834-bd4e-23562f88436d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.213229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.222613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 252/506] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:03.223392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e6102-cb9e-4834-bd4e-23562f88436d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.224715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35a9db-2d58-4fe7-b8ae-8a62d0f9f37b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.258456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae84f9-4d78-40b9-8165-e7f59e22c15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.259041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae84f9-4d78-40b9-8165-e7f59e22c15d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.259322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae84f9-4d78-40b9-8165-e7f59e22c15d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.259599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae84f9-4d78-40b9-8165-e7f59e22c15d None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.260315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae84f9-4d78-40b9-8165-e7f59e22c15d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:03.260621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae84f9-4d78-40b9-8165-e7f59e22c15d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.262009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae84f9-4d78-40b9-8165-e7f59e22c15d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.265791 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.273153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 255/507] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:03.273956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae84f9-4d78-40b9-8165-e7f59e22c15d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.321394 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e6102-cb9e-4834-bd4e-23562f88436d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.353447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5972f2-f772-4a0b-abc9-5b30c3d4ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.354131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5972f2-f772-4a0b-abc9-5b30c3d4ba9b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.354410 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5972f2-f772-4a0b-abc9-5b30c3d4ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.354680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5972f2-f772-4a0b-abc9-5b30c3d4ba9b None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.355500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5972f2-f772-4a0b-abc9-5b30c3d4ba9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:03.355804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5972f2-f772-4a0b-abc9-5b30c3d4ba9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.357219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5972f2-f772-4a0b-abc9-5b30c3d4ba9b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.372608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae84f9-4d78-40b9-8165-e7f59e22c15d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.393053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 253/508] 10.222.0.42 () {56 vars in 1137 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:03.394351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86b40-1303-43bd-aa68-ad7f3e20aee0 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.394637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86b40-1303-43bd-aa68-ad7f3e20aee0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.394909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86b40-1303-43bd-aa68-ad7f3e20aee0 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.396120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 254/509] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:03.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86b40-1303-43bd-aa68-ad7f3e20aee0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.410424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dce6-a730-4c41-96f6-4e1b35d79faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.411088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dce6-a730-4c41-96f6-4e1b35d79faf None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.411377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dce6-a730-4c41-96f6-4e1b35d79faf None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.411654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dce6-a730-4c41-96f6-4e1b35d79faf None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.412395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163dce6-a730-4c41-96f6-4e1b35d79faf None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:03.412702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163dce6-a730-4c41-96f6-4e1b35d79faf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.414639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163dce6-a730-4c41-96f6-4e1b35d79faf None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.422705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 256/510] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:03.423474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dce6-a730-4c41-96f6-4e1b35d79faf None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.436886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.492752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.496576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86b40-1303-43bd-aa68-ad7f3e20aee0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.520003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dce6-a730-4c41-96f6-4e1b35d79faf None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.529537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ba650-c404-4424-b2f2-78d1e604ac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.530356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ba650-c404-4424-b2f2-78d1e604ac43 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.530640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ba650-c404-4424-b2f2-78d1e604ac43 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.530915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ba650-c404-4424-b2f2-78d1e604ac43 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.539077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ba650-c404-4424-b2f2-78d1e604ac43 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.540775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ba650-c404-4424-b2f2-78d1e604ac43 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.553906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ea14b-aeac-4bc5-85e0-0881c0451a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.554687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ea14b-aeac-4bc5-85e0-0881c0451a30 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.554977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ea14b-aeac-4bc5-85e0-0881c0451a30 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-370ba650-c404-4424-b2f2-78d1e604ac43 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:03.555064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:03.555879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 09:38:03.556749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:03.557238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ea14b-aeac-4bc5-85e0-0881c0451a30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.557303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 255/511] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:03.557303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ba650-c404-4424-b2f2-78d1e604ac43 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.566205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ea14b-aeac-4bc5-85e0-0881c0451a30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.567655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ea14b-aeac-4bc5-85e0-0881c0451a30 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c26ea14b-aeac-4bc5-85e0-0881c0451a30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:03.583053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:03.583949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:03.585120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:03.585120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:03.585120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:03.585120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:03.585120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:03.585120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:03.585120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:03.585120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:03.585120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 257/512] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:03.585989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a0daf-95fe-4c1b-9a90-6c5679a038d0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.586082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a0daf-95fe-4c1b-9a90-6c5679a038d0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.586286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a0daf-95fe-4c1b-9a90-6c5679a038d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.602410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1f1a0daf-95fe-4c1b-9a90-6c5679a038d0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:03.603565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1f1a0daf-95fe-4c1b-9a90-6c5679a038d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:03.657381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ba650-c404-4424-b2f2-78d1e604ac43 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.665324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.670281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 258/513] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:03.671059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a0daf-95fe-4c1b-9a90-6c5679a038d0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.699595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a22e58-5285-4efd-a1f6-dd0ffe471fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.700244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a22e58-5285-4efd-a1f6-dd0ffe471fa1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.700539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a22e58-5285-4efd-a1f6-dd0ffe471fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.700816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a22e58-5285-4efd-a1f6-dd0ffe471fa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.731263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.767929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a0daf-95fe-4c1b-9a90-6c5679a038d0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.771668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a22e58-5285-4efd-a1f6-dd0ffe471fa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.773141 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a22e58-5285-4efd-a1f6-dd0ffe471fa1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.800557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8aeb72-dd2d-4289-8189-5b09f3a9ab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.801351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aeb72-dd2d-4289-8189-5b09f3a9ab9c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.801636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aeb72-dd2d-4289-8189-5b09f3a9ab9c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.801919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aeb72-dd2d-4289-8189-5b09f3a9ab9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.812138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8aeb72-dd2d-4289-8189-5b09f3a9ab9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.813581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8aeb72-dd2d-4289-8189-5b09f3a9ab9c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0c8aeb72-dd2d-4289-8189-5b09f3a9ab9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:03.830529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:03.831361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:03.832179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:03.832179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:03.832179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:03.832179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:03.832179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 259/514] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:03.832538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8aeb72-dd2d-4289-8189-5b09f3a9ab9c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.875182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 256/515] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:03.876009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a22e58-5285-4efd-a1f6-dd0ffe471fa1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:03.885271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.930744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8aeb72-dd2d-4289-8189-5b09f3a9ab9c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.934257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:03.964161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25533ac5-72ca-4763-a621-9687225564b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:03.964820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25533ac5-72ca-4763-a621-9687225564b1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:03.965107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25533ac5-72ca-4763-a621-9687225564b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:03.965385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25533ac5-72ca-4763-a621-9687225564b1 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:03.966242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25533ac5-72ca-4763-a621-9687225564b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:03.966553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25533ac5-72ca-4763-a621-9687225564b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:03.967982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25533ac5-72ca-4763-a621-9687225564b1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:03.973180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a22e58-5285-4efd-a1f6-dd0ffe471fa1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:03.984989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 260/516] 10.222.0.42 () {56 vars in 1137 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:03.985694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25533ac5-72ca-4763-a621-9687225564b1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.009238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0e03-9610-4791-8d15-e16a79d6c7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.009905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0e03-9610-4791-8d15-e16a79d6c7d7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.010196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0e03-9610-4791-8d15-e16a79d6c7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.010471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0e03-9610-4791-8d15-e16a79d6c7d7 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.011245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0e03-9610-4791-8d15-e16a79d6c7d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:04.011550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0e03-9610-4791-8d15-e16a79d6c7d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.012989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0e03-9610-4791-8d15-e16a79d6c7d7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.030570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 257/517] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:04.031290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0e03-9610-4791-8d15-e16a79d6c7d7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.086279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25533ac5-72ca-4763-a621-9687225564b1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.096596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:04.119431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8c7ff-2657-4e38-b3a2-7baedf773038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.120224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8c7ff-2657-4e38-b3a2-7baedf773038 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.120508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8c7ff-2657-4e38-b3a2-7baedf773038 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.120787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8c7ff-2657-4e38-b3a2-7baedf773038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.127502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0e03-9610-4791-8d15-e16a79d6c7d7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.130961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8c7ff-2657-4e38-b3a2-7baedf773038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.132985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8c7ff-2657-4e38-b3a2-7baedf773038 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.147442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-6ba8c7ff-2657-4e38-b3a2-7baedf773038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:04.150038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:04.150852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:04.151679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:04.151679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:04.151679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:04.151679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:04.151679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 261/518] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:04.152287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8c7ff-2657-4e38-b3a2-7baedf773038 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.166239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4464e-d5fd-42f4-acb3-5c53bb80fdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.166909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4464e-d5fd-42f4-acb3-5c53bb80fdc2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.167198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4464e-d5fd-42f4-acb3-5c53bb80fdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.167485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4464e-d5fd-42f4-acb3-5c53bb80fdc2 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.168285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4464e-d5fd-42f4-acb3-5c53bb80fdc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:04.168598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4464e-d5fd-42f4-acb3-5c53bb80fdc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.170018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4464e-d5fd-42f4-acb3-5c53bb80fdc2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.182341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 258/519] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:04.183029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4464e-d5fd-42f4-acb3-5c53bb80fdc2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.255798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8c7ff-2657-4e38-b3a2-7baedf773038 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.279299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4464e-d5fd-42f4-acb3-5c53bb80fdc2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.289557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6ca37-fce0-4ba6-9cb2-e7dae0b1e0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.290578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6ca37-fce0-4ba6-9cb2-e7dae0b1e0e4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.290881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6ca37-fce0-4ba6-9cb2-e7dae0b1e0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.291162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6ca37-fce0-4ba6-9cb2-e7dae0b1e0e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.300729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6ca37-fce0-4ba6-9cb2-e7dae0b1e0e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.302190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6ca37-fce0-4ba6-9cb2-e7dae0b1e0e4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.312664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf4427-365a-43b5-9cec-6523af891bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.313299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf4427-365a-43b5-9cec-6523af891bf0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.313586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf4427-365a-43b5-9cec-6523af891bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.313867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf4427-365a-43b5-9cec-6523af891bf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-48e6ca37-fce0-4ba6-9cb2-e7dae0b1e0e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:04.320419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:04.321300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:04.322165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:04.322165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:04.322165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:04.322165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:04.322165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 262/520] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:04.322426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6ca37-fce0-4ba6-9cb2-e7dae0b1e0e4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.337571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:04.367599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:04.386842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf4427-365a-43b5-9cec-6523af891bf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.388302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf4427-365a-43b5-9cec-6523af891bf0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.420947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6ca37-fce0-4ba6-9cb2-e7dae0b1e0e4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.454568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95edcf-a26a-491d-af23-e8666cd11793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.455205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95edcf-a26a-491d-af23-e8666cd11793 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.455482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95edcf-a26a-491d-af23-e8666cd11793 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.455754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95edcf-a26a-491d-af23-e8666cd11793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.456575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95edcf-a26a-491d-af23-e8666cd11793 None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=dfe8ef12db914a9693ef470f1706bb5a) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:04.456930 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95edcf-a26a-491d-af23-e8666cd11793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.458388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95edcf-a26a-491d-af23-e8666cd11793 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.463959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 263/521] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:04 2022] GET /identity/v3/role_assignments?user.id=dfe8ef12db914a9693ef470f1706bb5a&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:04.464750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95edcf-a26a-491d-af23-e8666cd11793 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.492816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 259/522] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:04.492816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf4427-365a-43b5-9cec-6523af891bf0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.561050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:04.580892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:04.588535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95edcf-a26a-491d-af23-e8666cd11793 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.602391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf4427-365a-43b5-9cec-6523af891bf0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.628971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cead8a-6247-4713-9bf0-56bea6ea1b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.629654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cead8a-6247-4713-9bf0-56bea6ea1b93 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.629987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cead8a-6247-4713-9bf0-56bea6ea1b93 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.630348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cead8a-6247-4713-9bf0-56bea6ea1b93 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.631095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cead8a-6247-4713-9bf0-56bea6ea1b93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:04.631396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cead8a-6247-4713-9bf0-56bea6ea1b93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.632968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cead8a-6247-4713-9bf0-56bea6ea1b93 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.644605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f7401-c27b-433e-a3f7-8611fa7a6eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.645929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 264/523] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:04.646301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cead8a-6247-4713-9bf0-56bea6ea1b93 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.650979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f7401-c27b-433e-a3f7-8611fa7a6eff None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.651268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f7401-c27b-433e-a3f7-8611fa7a6eff None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.651574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f7401-c27b-433e-a3f7-8611fa7a6eff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.660860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f7401-c27b-433e-a3f7-8611fa7a6eff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.663013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f7401-c27b-433e-a3f7-8611fa7a6eff None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-196f7401-c27b-433e-a3f7-8611fa7a6eff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:04.682401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:04.683333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 260/524] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:04.684821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f7401-c27b-433e-a3f7-8611fa7a6eff None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.765688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cead8a-6247-4713-9bf0-56bea6ea1b93 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.796590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f7401-c27b-433e-a3f7-8611fa7a6eff None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.805548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f72de7-6a18-443e-89d4-d7527ec78b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.806519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f72de7-6a18-443e-89d4-d7527ec78b43 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.806875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f72de7-6a18-443e-89d4-d7527ec78b43 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.807218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f72de7-6a18-443e-89d4-d7527ec78b43 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.808002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f72de7-6a18-443e-89d4-d7527ec78b43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:04.808374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f72de7-6a18-443e-89d4-d7527ec78b43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.809810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f72de7-6a18-443e-89d4-d7527ec78b43 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.817233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:04.822349 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 265/525] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:04.823143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f72de7-6a18-443e-89d4-d7527ec78b43 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.837735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58922a51-a1db-40ef-92fe-3b6978ac139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.838563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58922a51-a1db-40ef-92fe-3b6978ac139a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.838906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58922a51-a1db-40ef-92fe-3b6978ac139a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.839189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58922a51-a1db-40ef-92fe-3b6978ac139a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.847996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:04.850146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58922a51-a1db-40ef-92fe-3b6978ac139a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.851671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58922a51-a1db-40ef-92fe-3b6978ac139a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-58922a51-a1db-40ef-92fe-3b6978ac139a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:04.868947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:04.869817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:04.875567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:04.875567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:04.875567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:04.875567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:04.875567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 261/526] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:04.875567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58922a51-a1db-40ef-92fe-3b6978ac139a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:04.928591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f72de7-6a18-443e-89d4-d7527ec78b43 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:04.964775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062e1ac-d522-4cd0-9c10-d9d7a33fac20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:04.965452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062e1ac-d522-4cd0-9c10-d9d7a33fac20 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:04.965740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062e1ac-d522-4cd0-9c10-d9d7a33fac20 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:04.966058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062e1ac-d522-4cd0-9c10-d9d7a33fac20 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:04.966761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062e1ac-d522-4cd0-9c10-d9d7a33fac20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:04.968196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062e1ac-d522-4cd0-9c10-d9d7a33fac20 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:04.978836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58922a51-a1db-40ef-92fe-3b6978ac139a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:05.012493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083bab4-e796-484b-86f8-867fd9bd276d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:05.013159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083bab4-e796-484b-86f8-867fd9bd276d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:05.013442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083bab4-e796-484b-86f8-867fd9bd276d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:05.013717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083bab4-e796-484b-86f8-867fd9bd276d None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:05.014591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083bab4-e796-484b-86f8-867fd9bd276d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:05.014907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083bab4-e796-484b-86f8-867fd9bd276d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:05.016328 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083bab4-e796-484b-86f8-867fd9bd276d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:05.033649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 262/527] 10.222.0.42 () {56 vars in 1139 bytes} [Fri Jun 3 09:38:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:05.034483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083bab4-e796-484b-86f8-867fd9bd276d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:05.048353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 266/528] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:38:04 2022] POST /identity/v3/users => generated 247 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:05.049401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062e1ac-d522-4cd0-9c10-d9d7a33fac20 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:05.118182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:05.146294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:05.169999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083bab4-e796-484b-86f8-867fd9bd276d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:05.181964 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062e1ac-d522-4cd0-9c10-d9d7a33fac20 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:05.209097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee0fd9-6797-4ff7-b09f-bf7a754f0d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:05.210487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee0fd9-6797-4ff7-b09f-bf7a754f0d15 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:05.210986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee0fd9-6797-4ff7-b09f-bf7a754f0d15 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:05.211481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee0fd9-6797-4ff7-b09f-bf7a754f0d15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:05.219685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee0fd9-6797-4ff7-b09f-bf7a754f0d15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:05.221499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee0fd9-6797-4ff7-b09f-bf7a754f0d15 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:05.233951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014797-59c6-4a66-a015-5e3943a70573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:05.235011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014797-59c6-4a66-a015-5e3943a70573 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:05.235516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014797-59c6-4a66-a015-5e3943a70573 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:05.235993 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014797-59c6-4a66-a015-5e3943a70573 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:05.236935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89014797-59c6-4a66-a015-5e3943a70573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:05.237517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89014797-59c6-4a66-a015-5e3943a70573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:05.239238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89014797-59c6-4a66-a015-5e3943a70573 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-7cee0fd9-6797-4ff7-b09f-bf7a754f0d15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:05.241288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:05.242144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:05.242990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:05.242990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:05.242990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:05.242990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:05.242990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:05.242990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:05.242990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:05.242990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:05.246080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 263/529] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:05.247200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee0fd9-6797-4ff7-b09f-bf7a754f0d15 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:05.260447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 267/530] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:05.261686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014797-59c6-4a66-a015-5e3943a70573 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:05.374673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee0fd9-6797-4ff7-b09f-bf7a754f0d15 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:05.387633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:05.402979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014797-59c6-4a66-a015-5e3943a70573 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:05.417926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673e812-c460-42fb-a8da-aa960552dea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:05.419010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673e812-c460-42fb-a8da-aa960552dea0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:05.419396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673e812-c460-42fb-a8da-aa960552dea0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:05.420080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673e812-c460-42fb-a8da-aa960552dea0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:05.422801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:05.449080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9460-6326-4795-9491-2696e115515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:05.450288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9460-6326-4795-9491-2696e115515d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:05.450743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9460-6326-4795-9491-2696e115515d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:05.451180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9460-6326-4795-9491-2696e115515d None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:05.452880 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9460-6326-4795-9491-2696e115515d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:05.453471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9460-6326-4795-9491-2696e115515d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:05.455974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9460-6326-4795-9491-2696e115515d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:05.481427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 268/531] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:05.482538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9460-6326-4795-9491-2696e115515d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:05.515680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673e812-c460-42fb-a8da-aa960552dea0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:05.517567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673e812-c460-42fb-a8da-aa960552dea0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:05.615795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9460-6326-4795-9491-2696e115515d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:05.657637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 264/532] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:05.658792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673e812-c460-42fb-a8da-aa960552dea0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:05.666816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228368f-724a-41d5-8e65-dda3cd08288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:05.667868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228368f-724a-41d5-8e65-dda3cd08288a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:05.668348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228368f-724a-41d5-8e65-dda3cd08288a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:05.668775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228368f-724a-41d5-8e65-dda3cd08288a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:05.673333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:05.678555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228368f-724a-41d5-8e65-dda3cd08288a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:05.680688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228368f-724a-41d5-8e65-dda3cd08288a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c228368f-724a-41d5-8e65-dda3cd08288a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:05.699952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:05.700841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:05.701655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:05.701655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:05.701655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:05.701655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:05.701655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:05.701655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:05.701655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:05.701655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:05.701655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 269/533] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:05.708090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228368f-724a-41d5-8e65-dda3cd08288a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:05.765050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:05.784667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673e812-c460-42fb-a8da-aa960552dea0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:05.820776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69126a67-206f-404a-ab97-7ae920fdd176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:05.821552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69126a67-206f-404a-ab97-7ae920fdd176 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:05.821720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69126a67-206f-404a-ab97-7ae920fdd176 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:05.822056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69126a67-206f-404a-ab97-7ae920fdd176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:05.822887 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69126a67-206f-404a-ab97-7ae920fdd176 None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=d59a351971e94aa38a22e8113207e5b6) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:05.823195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69126a67-206f-404a-ab97-7ae920fdd176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:05.824630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69126a67-206f-404a-ab97-7ae920fdd176 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:05.830968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 265/534] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:05 2022] GET /identity/v3/role_assignments?user.id=d59a351971e94aa38a22e8113207e5b6&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:05.831838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69126a67-206f-404a-ab97-7ae920fdd176 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:05.860898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228368f-724a-41d5-8e65-dda3cd08288a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:05.917991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7030441-7709-4c66-b83e-e116a081eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:05.917991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030441-7709-4c66-b83e-e116a081eaf6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:05.917991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030441-7709-4c66-b83e-e116a081eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:05.917991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7030441-7709-4c66-b83e-e116a081eaf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:05.929914 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:05.936812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7030441-7709-4c66-b83e-e116a081eaf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:05.938378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7030441-7709-4c66-b83e-e116a081eaf6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-e7030441-7709-4c66-b83e-e116a081eaf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:05.979706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:05.980628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:05.981434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:05.981434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:05.981434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:05.981434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:05.983445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 270/535] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:05.985519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69126a67-206f-404a-ab97-7ae920fdd176 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:05.988499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7030441-7709-4c66-b83e-e116a081eaf6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:06.062028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ff564-8c7e-45a8-8485-a1b08128455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:06.062709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ff564-8c7e-45a8-8485-a1b08128455f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:06.062994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ff564-8c7e-45a8-8485-a1b08128455f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:06.063273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ff564-8c7e-45a8-8485-a1b08128455f None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:06.064070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ff564-8c7e-45a8-8485-a1b08128455f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:06.064383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ff564-8c7e-45a8-8485-a1b08128455f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:06.066493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ff564-8c7e-45a8-8485-a1b08128455f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:06.079645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 266/536] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:06.086593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ff564-8c7e-45a8-8485-a1b08128455f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:06.146227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:06.173372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7030441-7709-4c66-b83e-e116a081eaf6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:06.242489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d5d0-8970-4c00-bbb6-dbf8fea4c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:06.243157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d5d0-8970-4c00-bbb6-dbf8fea4c3e7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:06.243445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d5d0-8970-4c00-bbb6-dbf8fea4c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:06.243722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361d5d0-8970-4c00-bbb6-dbf8fea4c3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:06.244442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361d5d0-8970-4c00-bbb6-dbf8fea4c3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:06.244748 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361d5d0-8970-4c00-bbb6-dbf8fea4c3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:06.246255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361d5d0-8970-4c00-bbb6-dbf8fea4c3e7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:06.248285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ff564-8c7e-45a8-8485-a1b08128455f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:06.259417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 271/537] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:06.265142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d5d0-8970-4c00-bbb6-dbf8fea4c3e7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:06.313388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a4c2b-406d-4451-837f-dffbb591e17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:06.314101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a4c2b-406d-4451-837f-dffbb591e17c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:06.314363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a4c2b-406d-4451-837f-dffbb591e17c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:06.314641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a4c2b-406d-4451-837f-dffbb591e17c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:06.315476 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a4c2b-406d-4451-837f-dffbb591e17c None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=aee48298b3834984a12fe2fe0a6f9a77) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:06.315789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a4c2b-406d-4451-837f-dffbb591e17c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:06.317212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a4c2b-406d-4451-837f-dffbb591e17c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:06.320803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:06.334932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 267/538] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:06 2022] GET /identity/v3/role_assignments?user.id=aee48298b3834984a12fe2fe0a6f9a77&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:06.399185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361d5d0-8970-4c00-bbb6-dbf8fea4c3e7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:06.464985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29a7dc-80d7-4b22-8d4e-de6a3fefc3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:06.464985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29a7dc-80d7-4b22-8d4e-de6a3fefc3a3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:06.464985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29a7dc-80d7-4b22-8d4e-de6a3fefc3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:06.464985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29a7dc-80d7-4b22-8d4e-de6a3fefc3a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:06.468804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29a7dc-80d7-4b22-8d4e-de6a3fefc3a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:06.470361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29a7dc-80d7-4b22-8d4e-de6a3fefc3a3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7c29a7dc-80d7-4b22-8d4e-de6a3fefc3a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:06.490463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:06.491447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:06.492292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:06.492292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:06.492292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:06.492292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:06.492292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:06.492292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:06.492292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:06.492292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:06.496841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 272/539] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:06.497783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a4c2b-406d-4451-837f-dffbb591e17c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:06.646531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a4c2b-406d-4451-837f-dffbb591e17c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:06.682155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14b068-9c3d-4f51-9716-868fde1de273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:06.682823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b068-9c3d-4f51-9716-868fde1de273 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:06.683110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b068-9c3d-4f51-9716-868fde1de273 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:06.683399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14b068-9c3d-4f51-9716-868fde1de273 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:06.684191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14b068-9c3d-4f51-9716-868fde1de273 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:06.684505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14b068-9c3d-4f51-9716-868fde1de273 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:06.685961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14b068-9c3d-4f51-9716-868fde1de273 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:06.703751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 268/540] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:06.717764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29a7dc-80d7-4b22-8d4e-de6a3fefc3a3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:06.738280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:06.895036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc2abe-f61a-4905-9b8c-c583d9df5094 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:06.895589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc2abe-f61a-4905-9b8c-c583d9df5094 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:06.895589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc2abe-f61a-4905-9b8c-c583d9df5094 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:06.897763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 269/541] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:38:06.908711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29a7dc-80d7-4b22-8d4e-de6a3fefc3a3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:06.919308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a733b-7c03-4ac7-92c7-39465064b33e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:06.919598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a733b-7c03-4ac7-92c7-39465064b33e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:06.919886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a733b-7c03-4ac7-92c7-39465064b33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:06.948656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d781ba-78a1-439f-b252-15e15fc9fbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:06.949374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d781ba-78a1-439f-b252-15e15fc9fbfa None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:06.949661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d781ba-78a1-439f-b252-15e15fc9fbfa None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:06.949978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d781ba-78a1-439f-b252-15e15fc9fbfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:06.951001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d781ba-78a1-439f-b252-15e15fc9fbfa None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=362797538cbd48dc9c73b09b0548959e) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:06.951320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d781ba-78a1-439f-b252-15e15fc9fbfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:06.952761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d781ba-78a1-439f-b252-15e15fc9fbfa None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:06.958903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d51a733b-7c03-4ac7-92c7-39465064b33e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:06.959368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 273/542] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:06 2022] GET /identity/v3/role_assignments?user.id=362797538cbd48dc9c73b09b0548959e&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:06.960697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d51a733b-7c03-4ac7-92c7-39465064b33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:07.072954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:07.096290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 270/543] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:07.132199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d781ba-78a1-439f-b252-15e15fc9fbfa None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:07.217235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:07.250425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d781ba-78a1-439f-b252-15e15fc9fbfa None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:07.289762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9feefc4-e708-4143-bacf-5e460c7bc739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:07.290845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feefc4-e708-4143-bacf-5e460c7bc739 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:07.291124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feefc4-e708-4143-bacf-5e460c7bc739 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:07.291394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feefc4-e708-4143-bacf-5e460c7bc739 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:07.302029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9feefc4-e708-4143-bacf-5e460c7bc739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c9feefc4-e708-4143-bacf-5e460c7bc739 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:07.318310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:07.319790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:07.320556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 274/544] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:07.327015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a733b-7c03-4ac7-92c7-39465064b33e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:07.523452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a733b-7c03-4ac7-92c7-39465064b33e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:07.571935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6586f-686c-4444-a0a7-b70cd908ce7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:07.572992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6586f-686c-4444-a0a7-b70cd908ce7e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:07.573499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6586f-686c-4444-a0a7-b70cd908ce7e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:07.574006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6586f-686c-4444-a0a7-b70cd908ce7e None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:07.574975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6586f-686c-4444-a0a7-b70cd908ce7e None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:07.575481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6586f-686c-4444-a0a7-b70cd908ce7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:07.577107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6586f-686c-4444-a0a7-b70cd908ce7e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:07.589993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 271/545] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:07.595294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9feefc4-e708-4143-bacf-5e460c7bc739 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:07.730246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9feefc4-e708-4143-bacf-5e460c7bc739 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:07.768644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:07.779878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43bdac-aea6-4d60-a4a4-6b6a8a82a066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:07.780688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43bdac-aea6-4d60-a4a4-6b6a8a82a066 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:07.786469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43bdac-aea6-4d60-a4a4-6b6a8a82a066 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:07.786469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43bdac-aea6-4d60-a4a4-6b6a8a82a066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:07.792506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43bdac-aea6-4d60-a4a4-6b6a8a82a066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:07.793991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43bdac-aea6-4d60-a4a4-6b6a8a82a066 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:07.811797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a468a-d67c-45d5-8b24-9026a752f3ac None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:07.812133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a468a-d67c-45d5-8b24-9026a752f3ac None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:07.812364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a468a-d67c-45d5-8b24-9026a752f3ac None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7b43bdac-aea6-4d60-a4a4-6b6a8a82a066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:07.812669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:07.813460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:07.814751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:07.814751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:07.814751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:07.814751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:07.815457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 272/547] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:07.819075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 275/547] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:07.820727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43bdac-aea6-4d60-a4a4-6b6a8a82a066 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:07.820840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd0080-b898-4bd1-8398-d7c9e2eec2b6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:07.821103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd0080-b898-4bd1-8398-d7c9e2eec2b6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:07.821383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd0080-b898-4bd1-8398-d7c9e2eec2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:07.842131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6dcd0080-b898-4bd1-8398-d7c9e2eec2b6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:07.842131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6dcd0080-b898-4bd1-8398-d7c9e2eec2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:07.888886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:07.914047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 273/548] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:38:07.936592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd0080-b898-4bd1-8398-d7c9e2eec2b6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:07.996402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43bdac-aea6-4d60-a4a4-6b6a8a82a066 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:08.042659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd0080-b898-4bd1-8398-d7c9e2eec2b6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:08.063129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24161710-6535-4c0a-9b5a-5977d1874466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:08.064165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24161710-6535-4c0a-9b5a-5977d1874466 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.064735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24161710-6535-4c0a-9b5a-5977d1874466 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.065202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24161710-6535-4c0a-9b5a-5977d1874466 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.066146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24161710-6535-4c0a-9b5a-5977d1874466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:08.066648 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24161710-6535-4c0a-9b5a-5977d1874466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:08.068226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24161710-6535-4c0a-9b5a-5977d1874466 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:08.079229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c4333-84f4-4a05-9817-163b2fa65f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:08.080027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c4333-84f4-4a05-9817-163b2fa65f5f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.085972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c4333-84f4-4a05-9817-163b2fa65f5f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.085972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c4333-84f4-4a05-9817-163b2fa65f5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.088044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 276/549] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:08.092710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c4333-84f4-4a05-9817-163b2fa65f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:08.093711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24161710-6535-4c0a-9b5a-5977d1874466 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d95c4333-84f4-4a05-9817-163b2fa65f5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:08.100859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:08.101943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:08.102739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:08.107234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 274/550] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:08.113916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c4333-84f4-4a05-9817-163b2fa65f5f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:08.190246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:08.190480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:08.291421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c4333-84f4-4a05-9817-163b2fa65f5f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:08.306179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24161710-6535-4c0a-9b5a-5977d1874466 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:08.368387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123962e2-6d0c-4c3a-b1d6-1f2a4ce9be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:08.369113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123962e2-6d0c-4c3a-b1d6-1f2a4ce9be75 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.369398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123962e2-6d0c-4c3a-b1d6-1f2a4ce9be75 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.369676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123962e2-6d0c-4c3a-b1d6-1f2a4ce9be75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.377493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123962e2-6d0c-4c3a-b1d6-1f2a4ce9be75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:08.379315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123962e2-6d0c-4c3a-b1d6-1f2a4ce9be75 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:08.387144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add83701-4ecf-4ad9-a9af-be16688128c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:08.387829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83701-4ecf-4ad9-a9af-be16688128c4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.388125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83701-4ecf-4ad9-a9af-be16688128c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.388407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83701-4ecf-4ad9-a9af-be16688128c4 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.389165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add83701-4ecf-4ad9-a9af-be16688128c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:08.389474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add83701-4ecf-4ad9-a9af-be16688128c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:08.390871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add83701-4ecf-4ad9-a9af-be16688128c4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-123962e2-6d0c-4c3a-b1d6-1f2a4ce9be75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:08.395055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:08.395927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:08.396805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 277/551] 10.222.0.42 () {56 vars in 1093 bytes} [Fri Jun 3 09:38:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:08.403828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123962e2-6d0c-4c3a-b1d6-1f2a4ce9be75 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:08.408773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 275/552] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:08.414755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add83701-4ecf-4ad9-a9af-be16688128c4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:08.564324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add83701-4ecf-4ad9-a9af-be16688128c4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:08.577593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:08.609685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85557d-95f9-4cd7-a265-aee1d815687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:08.610513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85557d-95f9-4cd7-a265-aee1d815687d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.610799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85557d-95f9-4cd7-a265-aee1d815687d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.611078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85557d-95f9-4cd7-a265-aee1d815687d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.619516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123962e2-6d0c-4c3a-b1d6-1f2a4ce9be75 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:08.635085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85557d-95f9-4cd7-a265-aee1d815687d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:08.635672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:08.636572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85557d-95f9-4cd7-a265-aee1d815687d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:08.655535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9243c-1211-4a6e-9edc-9d4ee7901c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:08.656293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9243c-1211-4a6e-9edc-9d4ee7901c30 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.656649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9243c-1211-4a6e-9edc-9d4ee7901c30 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.656999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9243c-1211-4a6e-9edc-9d4ee7901c30 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.657910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9243c-1211-4a6e-9edc-9d4ee7901c30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:08.658350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9243c-1211-4a6e-9edc-9d4ee7901c30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:08.659998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9243c-1211-4a6e-9edc-9d4ee7901c30 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-1d85557d-95f9-4cd7-a265-aee1d815687d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:08.668612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:08.669473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:08.670351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:08.670351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:08.670351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:08.670351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:08.670351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 276/553] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:08.679718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85557d-95f9-4cd7-a265-aee1d815687d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:08.687816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 278/554] 10.222.0.42 () {56 vars in 1133 bytes} [Fri Jun 3 09:38:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:08.695198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9243c-1211-4a6e-9edc-9d4ee7901c30 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:08.844462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85557d-95f9-4cd7-a265-aee1d815687d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:08.848145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9243c-1211-4a6e-9edc-9d4ee7901c30 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:08.882555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a6429-4025-42c7-add8-446b7bb57c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:08.883466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a6429-4025-42c7-add8-446b7bb57c07 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.883659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358128eb-a4d6-4fde-870e-a1cc47795f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:08.884129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a6429-4025-42c7-add8-446b7bb57c07 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.884557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a6429-4025-42c7-add8-446b7bb57c07 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.884682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358128eb-a4d6-4fde-870e-a1cc47795f67 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.884980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358128eb-a4d6-4fde-870e-a1cc47795f67 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.885319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358128eb-a4d6-4fde-870e-a1cc47795f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.885867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a6429-4025-42c7-add8-446b7bb57c07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:08.886381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a6429-4025-42c7-add8-446b7bb57c07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:08.888028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a6429-4025-42c7-add8-446b7bb57c07 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:08.896112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358128eb-a4d6-4fde-870e-a1cc47795f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:08.897709 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358128eb-a4d6-4fde-870e-a1cc47795f67 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:08.902155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 277/555] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:08.903707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cce658-72db-4756-8d2c-ec977f233769 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.904153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cce658-72db-4756-8d2c-ec977f233769 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.904564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cce658-72db-4756-8d2c-ec977f233769 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.909954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 278/556] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:08.911616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dc783-a1ec-4e17-95d3-10a250e04c68 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:08.912073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dc783-a1ec-4e17-95d3-10a250e04c68 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:08.912501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11dc783-a1ec-4e17-95d3-10a250e04c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-358128eb-a4d6-4fde-870e-a1cc47795f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:08.922236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:08.923270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:08.924655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:08.924655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:08.924655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:08.924655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:08.924655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 279/557] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:08.924655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358128eb-a4d6-4fde-870e-a1cc47795f67 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:08.934152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c11dc783-a1ec-4e17-95d3-10a250e04c68 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:08.936081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c11dc783-a1ec-4e17-95d3-10a250e04c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:08.940561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:08.948155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:09.018087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 279/558] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:38:09.019433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dc783-a1ec-4e17-95d3-10a250e04c68 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.031315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358128eb-a4d6-4fde-870e-a1cc47795f67 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.068094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becd20b-7ead-4818-a962-b4bf0423e8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.068889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becd20b-7ead-4818-a962-b4bf0423e8cb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.069766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becd20b-7ead-4818-a962-b4bf0423e8cb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.070081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becd20b-7ead-4818-a962-b4bf0423e8cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.078579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3becd20b-7ead-4818-a962-b4bf0423e8cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:09.080023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3becd20b-7ead-4818-a962-b4bf0423e8cb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3becd20b-7ead-4818-a962-b4bf0423e8cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:09.094722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:09.095721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:09.096737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 280/559] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:09.097313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce708fa-52f8-4a4c-a9ca-8ecae4444ab8 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.098060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce708fa-52f8-4a4c-a9ca-8ecae4444ab8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.098060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce708fa-52f8-4a4c-a9ca-8ecae4444ab8 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.099198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 281/560] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:09.099783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce708fa-52f8-4a4c-a9ca-8ecae4444ab8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.164456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11dc783-a1ec-4e17-95d3-10a250e04c68 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.188841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be033e-dee3-4d46-9a42-e065876532ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.189589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be033e-dee3-4d46-9a42-e065876532ac None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.189875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be033e-dee3-4d46-9a42-e065876532ac None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.190180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be033e-dee3-4d46-9a42-e065876532ac None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.190462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:09.190967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be033e-dee3-4d46-9a42-e065876532ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:09.191280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be033e-dee3-4d46-9a42-e065876532ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:09.192659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be033e-dee3-4d46-9a42-e065876532ac None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:09.207903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 280/561] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:09.207903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694273a2-53f6-4743-87cd-5557600058d0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.207903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694273a2-53f6-4743-87cd-5557600058d0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.207903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694273a2-53f6-4743-87cd-5557600058d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.214027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:09.224651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-694273a2-53f6-4743-87cd-5557600058d0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:09.225872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-694273a2-53f6-4743-87cd-5557600058d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:09.226157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce708fa-52f8-4a4c-a9ca-8ecae4444ab8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.264520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8560a-aadd-44ac-a231-c1594c98ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.265434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8560a-aadd-44ac-a231-c1594c98ea72 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.265720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8560a-aadd-44ac-a231-c1594c98ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.266028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8560a-aadd-44ac-a231-c1594c98ea72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.281919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8560a-aadd-44ac-a231-c1594c98ea72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-97a8560a-aadd-44ac-a231-c1594c98ea72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:09.290202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:09.291234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:09.292084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 282/562] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:09.292788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8560a-aadd-44ac-a231-c1594c98ea72 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.314033 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 281/563] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:09.315687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694273a2-53f6-4743-87cd-5557600058d0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.408398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8560a-aadd-44ac-a231-c1594c98ea72 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.424148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694273a2-53f6-4743-87cd-5557600058d0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.437458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:09.442192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3b17e-5144-4f1a-a7a5-5826c1226a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.442864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3b17e-5144-4f1a-a7a5-5826c1226a57 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.443149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3b17e-5144-4f1a-a7a5-5826c1226a57 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.443425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3b17e-5144-4f1a-a7a5-5826c1226a57 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.444284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3b17e-5144-4f1a-a7a5-5826c1226a57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:09.444593 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3b17e-5144-4f1a-a7a5-5826c1226a57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:09.446046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3b17e-5144-4f1a-a7a5-5826c1226a57 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:09.463459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 283/564] 10.222.0.42 () {56 vars in 1135 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:09.463967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2a1a7-8069-4b70-880d-bf89d29247f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.464490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3b17e-5144-4f1a-a7a5-5826c1226a57 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.465173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2a1a7-8069-4b70-880d-bf89d29247f7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.465451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2a1a7-8069-4b70-880d-bf89d29247f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.465721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2a1a7-8069-4b70-880d-bf89d29247f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.469646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:09.476232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2a1a7-8069-4b70-880d-bf89d29247f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:09.477696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2a1a7-8069-4b70-880d-bf89d29247f7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-e3d2a1a7-8069-4b70-880d-bf89d29247f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:09.492038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:09.493100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 282/565] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:09.493982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2a1a7-8069-4b70-880d-bf89d29247f7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.616904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3b17e-5144-4f1a-a7a5-5826c1226a57 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.617149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2a1a7-8069-4b70-880d-bf89d29247f7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.649556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54c619-f6db-49ee-ab3c-77df0d20f577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.649751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aede48-137f-4167-8bbe-5983cc63a4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.650320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54c619-f6db-49ee-ab3c-77df0d20f577 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.650555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aede48-137f-4167-8bbe-5983cc63a4ab None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.650658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54c619-f6db-49ee-ab3c-77df0d20f577 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.650835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aede48-137f-4167-8bbe-5983cc63a4ab None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.650934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54c619-f6db-49ee-ab3c-77df0d20f577 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.651108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aede48-137f-4167-8bbe-5983cc63a4ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.651607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54c619-f6db-49ee-ab3c-77df0d20f577 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:09.651915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54c619-f6db-49ee-ab3c-77df0d20f577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:09.653285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54c619-f6db-49ee-ab3c-77df0d20f577 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:09.660817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aede48-137f-4167-8bbe-5983cc63a4ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:09.661207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 284/566] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:09.662092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54c619-f6db-49ee-ab3c-77df0d20f577 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-93aede48-137f-4167-8bbe-5983cc63a4ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:09.669299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:09.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 283/567] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:09.671284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aede48-137f-4167-8bbe-5983cc63a4ab None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.716135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:09.730025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:09.763929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54c619-f6db-49ee-ab3c-77df0d20f577 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.773354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aede48-137f-4167-8bbe-5983cc63a4ab None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.798578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4fcf6-e286-4b89-87f6-4cf75a3bff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.799216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4fcf6-e286-4b89-87f6-4cf75a3bff27 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.799499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4fcf6-e286-4b89-87f6-4cf75a3bff27 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.799775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4fcf6-e286-4b89-87f6-4cf75a3bff27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.807203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263997dc-f1fa-4148-9c27-013736823e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.807857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263997dc-f1fa-4148-9c27-013736823e30 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.808147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263997dc-f1fa-4148-9c27-013736823e30 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.808435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263997dc-f1fa-4148-9c27-013736823e30 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.809221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263997dc-f1fa-4148-9c27-013736823e30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:09.809538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263997dc-f1fa-4148-9c27-013736823e30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:09.811024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263997dc-f1fa-4148-9c27-013736823e30 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:09.822876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 284/568] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:09.824095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db5a3-4755-4148-8e6a-839dd0c91c6f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.824385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db5a3-4755-4148-8e6a-839dd0c91c6f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.824663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4db5a3-4755-4148-8e6a-839dd0c91c6f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.826042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 285/569] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:09.830581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4db5a3-4755-4148-8e6a-839dd0c91c6f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.868549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4fcf6-e286-4b89-87f6-4cf75a3bff27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:09.870038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4fcf6-e286-4b89-87f6-4cf75a3bff27 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:09.920244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:09.930981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4db5a3-4755-4148-8e6a-839dd0c91c6f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:09.944685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:09.964447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa6e60-d27f-4427-b16a-f710f7eed26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:09.965219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa6e60-d27f-4427-b16a-f710f7eed26f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:09.965547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa6e60-d27f-4427-b16a-f710f7eed26f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:09.965769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa6e60-d27f-4427-b16a-f710f7eed26f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:09.972471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 285/570] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:09.972471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4fcf6-e286-4b89-87f6-4cf75a3bff27 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:09.976371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa6e60-d27f-4427-b16a-f710f7eed26f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:09.977815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa6e60-d27f-4427-b16a-f710f7eed26f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-adfa6e60-d27f-4427-b16a-f710f7eed26f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:09.995203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:09.996103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:09.996960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:09.996960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:09.996960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:09.996960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:09.997591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 286/571] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:09.998415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa6e60-d27f-4427-b16a-f710f7eed26f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.065487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4fcf6-e286-4b89-87f6-4cf75a3bff27 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.096326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa6e60-d27f-4427-b16a-f710f7eed26f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.099264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21dad7-32f3-4c93-92f9-1a7ba838e382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.099928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21dad7-32f3-4c93-92f9-1a7ba838e382 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.100807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21dad7-32f3-4c93-92f9-1a7ba838e382 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.101090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21dad7-32f3-4c93-92f9-1a7ba838e382 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.101833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21dad7-32f3-4c93-92f9-1a7ba838e382 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:10.102170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21dad7-32f3-4c93-92f9-1a7ba838e382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:10.103539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21dad7-32f3-4c93-92f9-1a7ba838e382 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:10.111238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 286/572] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:10.112402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0b472-b7f6-43e1-a449-a5f8dc01d398 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.112694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0b472-b7f6-43e1-a449-a5f8dc01d398 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.112977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0b472-b7f6-43e1-a449-a5f8dc01d398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.128897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7ee0b472-b7f6-43e1-a449-a5f8dc01d398 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:10.130108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7ee0b472-b7f6-43e1-a449-a5f8dc01d398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:10.130896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d09c2f-d84c-46cb-b8d4-1bab145dcd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.131859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d09c2f-d84c-46cb-b8d4-1bab145dcd24 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.132145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d09c2f-d84c-46cb-b8d4-1bab145dcd24 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.132417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d09c2f-d84c-46cb-b8d4-1bab145dcd24 None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/dfe8ef12db914a9693ef470f1706bb5a/roles/4766b6a3fe4746798f255d0813534a04` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.139506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:10.146502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d09c2f-d84c-46cb-b8d4-1bab145dcd24 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=dfe8ef12db914a9693ef470f1706bb5a, role_id=4766b6a3fe4746798f255d0813534a04)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:10.148506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d09c2f-d84c-46cb-b8d4-1bab145dcd24 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:10.195358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 287/573] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:10.196171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0b472-b7f6-43e1-a449-a5f8dc01d398 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.237307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 287/574] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:38:09 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/dfe8ef12db914a9693ef470f1706bb5a/roles/4766b6a3fe4746798f255d0813534a04 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:38:10.238129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d09c2f-d84c-46cb-b8d4-1bab145dcd24 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.274964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:10.301292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0b472-b7f6-43e1-a449-a5f8dc01d398 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.342000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60a76b-c65c-4207-81a7-b9bbf4a489af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.342836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60a76b-c65c-4207-81a7-b9bbf4a489af None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.343119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60a76b-c65c-4207-81a7-b9bbf4a489af None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.343396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60a76b-c65c-4207-81a7-b9bbf4a489af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.354643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60a76b-c65c-4207-81a7-b9bbf4a489af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:10.356138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60a76b-c65c-4207-81a7-b9bbf4a489af None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:10.359508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:10.364877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d09c2f-d84c-46cb-b8d4-1bab145dcd24 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-cf60a76b-c65c-4207-81a7-b9bbf4a489af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:10.376876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:10.378109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:10.379186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:10.379186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:10.379186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:10.379186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:10.383790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 288/575] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:10.384521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60a76b-c65c-4207-81a7-b9bbf4a489af None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.408574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22c07d-b140-4d0a-9a31-1699efb29bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.409255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22c07d-b140-4d0a-9a31-1699efb29bb4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.409539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22c07d-b140-4d0a-9a31-1699efb29bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.409823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22c07d-b140-4d0a-9a31-1699efb29bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.410597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22c07d-b140-4d0a-9a31-1699efb29bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:10.410910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22c07d-b140-4d0a-9a31-1699efb29bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:10.412333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22c07d-b140-4d0a-9a31-1699efb29bb4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:10.429849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 288/576] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:10.430838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22c07d-b140-4d0a-9a31-1699efb29bb4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.498189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60a76b-c65c-4207-81a7-b9bbf4a489af None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.518313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:10.536316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24898-30d3-4d54-b555-051f3b9f646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.537234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24898-30d3-4d54-b555-051f3b9f646b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.537770 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24898-30d3-4d54-b555-051f3b9f646b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.538080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df24898-30d3-4d54-b555-051f3b9f646b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.541325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22c07d-b140-4d0a-9a31-1699efb29bb4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.551014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df24898-30d3-4d54-b555-051f3b9f646b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:10.553145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df24898-30d3-4d54-b555-051f3b9f646b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-9df24898-30d3-4d54-b555-051f3b9f646b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:10.572708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:10.573800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:10.574928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:10.574928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:10.574928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:10.574928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:10.575785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 289/577] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:10.576613 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24898-30d3-4d54-b555-051f3b9f646b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.585226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81885d-fabd-412c-93a2-f7f2f8d9533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.586006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81885d-fabd-412c-93a2-f7f2f8d9533d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.586308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81885d-fabd-412c-93a2-f7f2f8d9533d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.586563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81885d-fabd-412c-93a2-f7f2f8d9533d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.598031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81885d-fabd-412c-93a2-f7f2f8d9533d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ac81885d-fabd-412c-93a2-f7f2f8d9533d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:10.605948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:10.606892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:10.607726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 289/578] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:10.608414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81885d-fabd-412c-93a2-f7f2f8d9533d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.634143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:10.688316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df24898-30d3-4d54-b555-051f3b9f646b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.723006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81885d-fabd-412c-93a2-f7f2f8d9533d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.724111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38286ad9-f606-4a6b-89c5-263bbcffca67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.724783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38286ad9-f606-4a6b-89c5-263bbcffca67 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.725075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38286ad9-f606-4a6b-89c5-263bbcffca67 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.726658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38286ad9-f606-4a6b-89c5-263bbcffca67 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.726658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38286ad9-f606-4a6b-89c5-263bbcffca67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:10.726962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38286ad9-f606-4a6b-89c5-263bbcffca67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:10.728341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38286ad9-f606-4a6b-89c5-263bbcffca67 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:10.740176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 290/579] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:10.741026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38286ad9-f606-4a6b-89c5-263bbcffca67 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.765776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd754f5-94e2-4292-8cf9-7c93763f3ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.766525 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd754f5-94e2-4292-8cf9-7c93763f3ef1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.766807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd754f5-94e2-4292-8cf9-7c93763f3ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.767083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd754f5-94e2-4292-8cf9-7c93763f3ef1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.770190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:10.776078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd754f5-94e2-4292-8cf9-7c93763f3ef1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:10.777518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd754f5-94e2-4292-8cf9-7c93763f3ef1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-fbd754f5-94e2-4292-8cf9-7c93763f3ef1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:10.791866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:10.792848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 290/580] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:10.793797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd754f5-94e2-4292-8cf9-7c93763f3ef1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.859262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38286ad9-f606-4a6b-89c5-263bbcffca67 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.871706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:10.897373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5120c-7d0c-4472-8b61-db7d7ffc5640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.898106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5120c-7d0c-4472-8b61-db7d7ffc5640 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.898401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5120c-7d0c-4472-8b61-db7d7ffc5640 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.898680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5120c-7d0c-4472-8b61-db7d7ffc5640 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.899395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5120c-7d0c-4472-8b61-db7d7ffc5640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:10.899703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5120c-7d0c-4472-8b61-db7d7ffc5640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:10.899815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd754f5-94e2-4292-8cf9-7c93763f3ef1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:10.901067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5120c-7d0c-4472-8b61-db7d7ffc5640 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:10.914347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 291/581] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:10.915721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5120c-7d0c-4472-8b61-db7d7ffc5640 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:10.942697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21897042-34fe-4e58-bb22-8bf2da555a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:10.943677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21897042-34fe-4e58-bb22-8bf2da555a77 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:10.944180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21897042-34fe-4e58-bb22-8bf2da555a77 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:10.944652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21897042-34fe-4e58-bb22-8bf2da555a77 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:10.945572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21897042-34fe-4e58-bb22-8bf2da555a77 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:10.946485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21897042-34fe-4e58-bb22-8bf2da555a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:10.948016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21897042-34fe-4e58-bb22-8bf2da555a77 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:10.958133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 291/582] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:10.959403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21897042-34fe-4e58-bb22-8bf2da555a77 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.018650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5120c-7d0c-4472-8b61-db7d7ffc5640 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.024898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:11.055582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc369dd-3058-4ba5-86e3-dea60674d1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.056373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc369dd-3058-4ba5-86e3-dea60674d1e0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.056912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc369dd-3058-4ba5-86e3-dea60674d1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.056912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc369dd-3058-4ba5-86e3-dea60674d1e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.063997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc369dd-3058-4ba5-86e3-dea60674d1e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.065351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc369dd-3058-4ba5-86e3-dea60674d1e0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.070084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21897042-34fe-4e58-bb22-8bf2da555a77 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-5dc369dd-3058-4ba5-86e3-dea60674d1e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:11.079310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:11.080682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:11.081574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:11.081574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:11.081574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:11.081574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:11.081574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:11.081574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:11.081574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:11.081574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:11.081574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 292/583] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:11.082095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc369dd-3058-4ba5-86e3-dea60674d1e0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.111778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abfd19-5940-4440-8264-2d795b72dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.112507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abfd19-5940-4440-8264-2d795b72dd98 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.112793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abfd19-5940-4440-8264-2d795b72dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.113075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abfd19-5940-4440-8264-2d795b72dd98 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.113569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:11.113979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abfd19-5940-4440-8264-2d795b72dd98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:11.114291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abfd19-5940-4440-8264-2d795b72dd98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.115802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abfd19-5940-4440-8264-2d795b72dd98 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.135123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 292/584] 10.222.0.42 () {56 vars in 1137 bytes} [Fri Jun 3 09:38:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:11.136001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abfd19-5940-4440-8264-2d795b72dd98 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.195727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc369dd-3058-4ba5-86e3-dea60674d1e0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.230356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5220c-561a-4dcc-9c56-5579b17e73ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.231148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5220c-561a-4dcc-9c56-5579b17e73ab None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.231436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5220c-561a-4dcc-9c56-5579b17e73ab None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.231710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5220c-561a-4dcc-9c56-5579b17e73ab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.238117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abfd19-5940-4440-8264-2d795b72dd98 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.239327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5220c-561a-4dcc-9c56-5579b17e73ab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.241480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5220c-561a-4dcc-9c56-5579b17e73ab None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-cbf5220c-561a-4dcc-9c56-5579b17e73ab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:11.255471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:11.256615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:11.257652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 293/585] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:11.258781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:11.261857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5220c-561a-4dcc-9c56-5579b17e73ab None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.282404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebab220-8034-4392-b605-15c4c3ee7861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.283236 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebab220-8034-4392-b605-15c4c3ee7861 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.283587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebab220-8034-4392-b605-15c4c3ee7861 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.283938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebab220-8034-4392-b605-15c4c3ee7861 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.296585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebab220-8034-4392-b605-15c4c3ee7861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.298145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebab220-8034-4392-b605-15c4c3ee7861 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-7ebab220-8034-4392-b605-15c4c3ee7861 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:11.317005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:11.317906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:11.318773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:11.318773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:11.318773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:11.318773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:11.318773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 293/586] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:11.319105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebab220-8034-4392-b605-15c4c3ee7861 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.363691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:11.376332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5220c-561a-4dcc-9c56-5579b17e73ab None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.416071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3d827-3023-408c-87a4-8cd1220ba39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.416832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3d827-3023-408c-87a4-8cd1220ba39f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.417188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3d827-3023-408c-87a4-8cd1220ba39f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.417552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3d827-3023-408c-87a4-8cd1220ba39f None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.418457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3d827-3023-408c-87a4-8cd1220ba39f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:11.418832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3d827-3023-408c-87a4-8cd1220ba39f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.420341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3d827-3023-408c-87a4-8cd1220ba39f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.424742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebab220-8034-4392-b605-15c4c3ee7861 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.437627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 294/587] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:11.437627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3d827-3023-408c-87a4-8cd1220ba39f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.458695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d440b42-5003-483a-87ad-42a3a1cf5ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.460021 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d440b42-5003-483a-87ad-42a3a1cf5ce0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.460297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d440b42-5003-483a-87ad-42a3a1cf5ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.460578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d440b42-5003-483a-87ad-42a3a1cf5ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.470657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d440b42-5003-483a-87ad-42a3a1cf5ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.472097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d440b42-5003-483a-87ad-42a3a1cf5ce0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-1d440b42-5003-483a-87ad-42a3a1cf5ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:11.491363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:11.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:11.493147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:11.493147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:11.493147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:11.493147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:11.497089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 294/588] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:11.497963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d440b42-5003-483a-87ad-42a3a1cf5ce0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.520917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:11.555377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3d827-3023-408c-87a4-8cd1220ba39f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.598297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705968fe-ef8e-425b-98ab-86303f36520a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.599064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705968fe-ef8e-425b-98ab-86303f36520a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.599352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705968fe-ef8e-425b-98ab-86303f36520a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.599558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705968fe-ef8e-425b-98ab-86303f36520a None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.600988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705968fe-ef8e-425b-98ab-86303f36520a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:11.601310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705968fe-ef8e-425b-98ab-86303f36520a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.602778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705968fe-ef8e-425b-98ab-86303f36520a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.604984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d440b42-5003-483a-87ad-42a3a1cf5ce0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.624873 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 295/589] 10.222.0.42 () {56 vars in 1137 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:11.624873 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705968fe-ef8e-425b-98ab-86303f36520a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.633225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:11.644533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b85707-a9d1-46e2-a61e-bf90291699a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.647901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b85707-a9d1-46e2-a61e-bf90291699a8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.647901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b85707-a9d1-46e2-a61e-bf90291699a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.647901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b85707-a9d1-46e2-a61e-bf90291699a8 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.647901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b85707-a9d1-46e2-a61e-bf90291699a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:11.647901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b85707-a9d1-46e2-a61e-bf90291699a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.648709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b85707-a9d1-46e2-a61e-bf90291699a8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.660830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 295/590] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:11.661751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b85707-a9d1-46e2-a61e-bf90291699a8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.740175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705968fe-ef8e-425b-98ab-86303f36520a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.747564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:11.771809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b85707-a9d1-46e2-a61e-bf90291699a8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.775905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5525b4f-2971-4a11-b438-7242ff870ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.776562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5525b4f-2971-4a11-b438-7242ff870ea2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.776847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5525b4f-2971-4a11-b438-7242ff870ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.777324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5525b4f-2971-4a11-b438-7242ff870ea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.778023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5525b4f-2971-4a11-b438-7242ff870ea2 None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=37186dc0010247cd936f160b5eb0298b) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:11.778337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5525b4f-2971-4a11-b438-7242ff870ea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.779766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5525b4f-2971-4a11-b438-7242ff870ea2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.785375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 296/591] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/role_assignments?user.id=37186dc0010247cd936f160b5eb0298b&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:11.786189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5525b4f-2971-4a11-b438-7242ff870ea2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.832302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688e453-01f4-47bd-b776-46aaf69da0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.833275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e453-01f4-47bd-b776-46aaf69da0e8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.833774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e453-01f4-47bd-b776-46aaf69da0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.834232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e453-01f4-47bd-b776-46aaf69da0e8 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.835946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688e453-01f4-47bd-b776-46aaf69da0e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:11.836459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688e453-01f4-47bd-b776-46aaf69da0e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.839038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688e453-01f4-47bd-b776-46aaf69da0e8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.859916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 296/592] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:11.859916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688e453-01f4-47bd-b776-46aaf69da0e8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:11.880564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:11.916600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5525b4f-2971-4a11-b438-7242ff870ea2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:11.955675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e136a0c-f817-438e-982e-fa0b6c92743f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:11.956591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e136a0c-f817-438e-982e-fa0b6c92743f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:11.956953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e136a0c-f817-438e-982e-fa0b6c92743f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:11.957303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e136a0c-f817-438e-982e-fa0b6c92743f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:11.972694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e136a0c-f817-438e-982e-fa0b6c92743f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:11.975146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e136a0c-f817-438e-982e-fa0b6c92743f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0e136a0c-f817-438e-982e-fa0b6c92743f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:11.999108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:12.000245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:12.006117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:12.006117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:12.006117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:12.006117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:12.006117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 297/593] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:12.006117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e136a0c-f817-438e-982e-fa0b6c92743f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:12.017398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688e453-01f4-47bd-b776-46aaf69da0e8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.055027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360ffba-7143-4a11-9d38-d55c47a27a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:12.055831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360ffba-7143-4a11-9d38-d55c47a27a5f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.056116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360ffba-7143-4a11-9d38-d55c47a27a5f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.056449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360ffba-7143-4a11-9d38-d55c47a27a5f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.064663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360ffba-7143-4a11-9d38-d55c47a27a5f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:12.066160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360ffba-7143-4a11-9d38-d55c47a27a5f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:12.070004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d360ffba-7143-4a11-9d38-d55c47a27a5f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:12.091451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:12.092413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 09:38:12.093337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:12.095275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 297/594] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:12.096311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360ffba-7143-4a11-9d38-d55c47a27a5f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:12.139066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e136a0c-f817-438e-982e-fa0b6c92743f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.172393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:12.189925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b40683-b523-4d53-8800-e66b59a1424e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:12.191224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b40683-b523-4d53-8800-e66b59a1424e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.191658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b40683-b523-4d53-8800-e66b59a1424e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.192541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b40683-b523-4d53-8800-e66b59a1424e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.201705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b40683-b523-4d53-8800-e66b59a1424e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:12.204126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b40683-b523-4d53-8800-e66b59a1424e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:12.212330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360ffba-7143-4a11-9d38-d55c47a27a5f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c3b40683-b523-4d53-8800-e66b59a1424e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:12.222989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:12.224047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:12.225164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:12.225164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:12.225164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:12.225164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:12.225164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:12.225164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:12.225164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:12.225164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:12.228426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 298/595] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:12.229549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b40683-b523-4d53-8800-e66b59a1424e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:12.252891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4a21a-c506-48fb-9058-9831f4c60e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:12.253754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4a21a-c506-48fb-9058-9831f4c60e74 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.254229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4a21a-c506-48fb-9058-9831f4c60e74 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.254672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4a21a-c506-48fb-9058-9831f4c60e74 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.255481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4a21a-c506-48fb-9058-9831f4c60e74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:12.255986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4a21a-c506-48fb-9058-9831f4c60e74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:12.257532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4a21a-c506-48fb-9058-9831f4c60e74 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:12.270589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 298/596] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:12.271726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4a21a-c506-48fb-9058-9831f4c60e74 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:12.337390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:12.367413 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b40683-b523-4d53-8800-e66b59a1424e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.390327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4a21a-c506-48fb-9058-9831f4c60e74 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.406473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07d6d1-59ee-417a-92dc-7477fc4dc1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:12.407275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07d6d1-59ee-417a-92dc-7477fc4dc1fc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.407665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07d6d1-59ee-417a-92dc-7477fc4dc1fc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.408108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07d6d1-59ee-417a-92dc-7477fc4dc1fc None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.409219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07d6d1-59ee-417a-92dc-7477fc4dc1fc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:12.410012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07d6d1-59ee-417a-92dc-7477fc4dc1fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:12.411350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07d6d1-59ee-417a-92dc-7477fc4dc1fc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:12.434503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e9614-44de-47fe-b73d-d3b93b91e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:12.435390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e9614-44de-47fe-b73d-d3b93b91e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.435779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 299/597] 10.222.0.42 () {56 vars in 1139 bytes} [Fri Jun 3 09:38:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:12.436080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e9614-44de-47fe-b73d-d3b93b91e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.436507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e9614-44de-47fe-b73d-d3b93b91e0f0 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.437454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e9614-44de-47fe-b73d-d3b93b91e0f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:12.437671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07d6d1-59ee-417a-92dc-7477fc4dc1fc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:12.438089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e9614-44de-47fe-b73d-d3b93b91e0f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:12.439812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e9614-44de-47fe-b73d-d3b93b91e0f0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:12.462168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 299/598] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:12.462168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e9614-44de-47fe-b73d-d3b93b91e0f0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:12.491318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:12.550686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07d6d1-59ee-417a-92dc-7477fc4dc1fc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.573752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e9614-44de-47fe-b73d-d3b93b91e0f0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.587292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969d1ee-a182-4da3-9634-8d0e3e88c3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:12.588217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969d1ee-a182-4da3-9634-8d0e3e88c3bc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.588499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969d1ee-a182-4da3-9634-8d0e3e88c3bc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.588830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969d1ee-a182-4da3-9634-8d0e3e88c3bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.594317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:12.596628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969d1ee-a182-4da3-9634-8d0e3e88c3bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:12.598111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969d1ee-a182-4da3-9634-8d0e3e88c3bc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:12.609103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d32565-2c47-42a3-8985-aeacc750d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:12.609952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d32565-2c47-42a3-8985-aeacc750d117 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.610244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d32565-2c47-42a3-8985-aeacc750d117 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.610523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d32565-2c47-42a3-8985-aeacc750d117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-5969d1ee-a182-4da3-9634-8d0e3e88c3bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:12.612439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:12.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 300/599] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:12.614497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969d1ee-a182-4da3-9634-8d0e3e88c3bc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:12.626324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d32565-2c47-42a3-8985-aeacc750d117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:12.627721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d32565-2c47-42a3-8985-aeacc750d117 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d1d32565-2c47-42a3-8985-aeacc750d117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:12.646823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:12.647796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:12.648706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:12.648706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:12.648706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:12.648706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:12.648706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 300/600] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:12.649463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c1a11-ac0a-40c0-b4b7-0c7f35179a41 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.649572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c1a11-ac0a-40c0-b4b7-0c7f35179a41 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.649863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c1a11-ac0a-40c0-b4b7-0c7f35179a41 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.654020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 301/601] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:12.654020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c1a11-ac0a-40c0-b4b7-0c7f35179a41 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:12.729412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:12.732400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969d1ee-a182-4da3-9634-8d0e3e88c3bc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.766447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a1d89-e000-4dca-9a53-704f60ab2b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:12.767337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a1d89-e000-4dca-9a53-704f60ab2b32 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.767854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a1d89-e000-4dca-9a53-704f60ab2b32 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.768179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a1d89-e000-4dca-9a53-704f60ab2b32 None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/d59a351971e94aa38a22e8113207e5b6/roles/4766b6a3fe4746798f255d0813534a04` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.773651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c1a11-ac0a-40c0-b4b7-0c7f35179a41 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.783840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a1d89-e000-4dca-9a53-704f60ab2b32 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=d59a351971e94aa38a22e8113207e5b6, role_id=4766b6a3fe4746798f255d0813534a04)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:12.785292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a1d89-e000-4dca-9a53-704f60ab2b32 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:12.809197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616d37a-2e87-42a6-bf0f-28e33f1238ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:12.809845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616d37a-2e87-42a6-bf0f-28e33f1238ce None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.810152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616d37a-2e87-42a6-bf0f-28e33f1238ce None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.810427 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616d37a-2e87-42a6-bf0f-28e33f1238ce None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.811224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616d37a-2e87-42a6-bf0f-28e33f1238ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:12.811529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616d37a-2e87-42a6-bf0f-28e33f1238ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:12.812930 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616d37a-2e87-42a6-bf0f-28e33f1238ce None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:12.825071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 302/602] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:12.831555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616d37a-2e87-42a6-bf0f-28e33f1238ce None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:12.854370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 301/603] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:38:12 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/d59a351971e94aa38a22e8113207e5b6/roles/4766b6a3fe4746798f255d0813534a04 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:38:12.855933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f7f43-f8f1-4c8e-945f-2f92875ddd15 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:12.860557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f7f43-f8f1-4c8e-945f-2f92875ddd15 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:12.860557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f7f43-f8f1-4c8e-945f-2f92875ddd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:12.879055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-989f7f43-f8f1-4c8e-945f-2f92875ddd15 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:12.881148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-989f7f43-f8f1-4c8e-945f-2f92875ddd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:12.887212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:12.956598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616d37a-2e87-42a6-bf0f-28e33f1238ce None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:12.979145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 302/604] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:12.980376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f7f43-f8f1-4c8e-945f-2f92875ddd15 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:13.020831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175607d9-2503-4bf3-9396-4fdc819d7ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:13.021518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175607d9-2503-4bf3-9396-4fdc819d7ecd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:13.021807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175607d9-2503-4bf3-9396-4fdc819d7ecd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:13.022694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175607d9-2503-4bf3-9396-4fdc819d7ecd None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:13.023567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175607d9-2503-4bf3-9396-4fdc819d7ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:13.023886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175607d9-2503-4bf3-9396-4fdc819d7ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:13.025323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175607d9-2503-4bf3-9396-4fdc819d7ecd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:13.039984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 303/605] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:13.045027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175607d9-2503-4bf3-9396-4fdc819d7ecd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:13.079189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:13.107199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f7f43-f8f1-4c8e-945f-2f92875ddd15 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:13.151592 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0326f3-8cd7-4378-addf-d1df5e35ccb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:13.152415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0326f3-8cd7-4378-addf-d1df5e35ccb2 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:13.152693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0326f3-8cd7-4378-addf-d1df5e35ccb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:13.152963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0326f3-8cd7-4378-addf-d1df5e35ccb2 None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/aee48298b3834984a12fe2fe0a6f9a77/roles/4766b6a3fe4746798f255d0813534a04` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:13.157004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:13.168303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0326f3-8cd7-4378-addf-d1df5e35ccb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=aee48298b3834984a12fe2fe0a6f9a77, role_id=4766b6a3fe4746798f255d0813534a04)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:13.170418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0326f3-8cd7-4378-addf-d1df5e35ccb2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:13.175902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175607d9-2503-4bf3-9396-4fdc819d7ecd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:13.211267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e65a86-9a60-46ee-9180-0a5ed956cf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:13.212145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e65a86-9a60-46ee-9180-0a5ed956cf0b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:13.212427 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e65a86-9a60-46ee-9180-0a5ed956cf0b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:13.212696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e65a86-9a60-46ee-9180-0a5ed956cf0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:13.223099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e65a86-9a60-46ee-9180-0a5ed956cf0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a3e65a86-9a60-46ee-9180-0a5ed956cf0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:13.231307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:13.232225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:13.233072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 304/606] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:13.233755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e65a86-9a60-46ee-9180-0a5ed956cf0b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:13.276128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 303/607] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:38:12 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/aee48298b3834984a12fe2fe0a6f9a77/roles/4766b6a3fe4746798f255d0813534a04 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:38:13.281462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0326f3-8cd7-4378-addf-d1df5e35ccb2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:13.402931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e65a86-9a60-46ee-9180-0a5ed956cf0b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:13.443023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26e284-1869-422a-b882-aa6a194c91e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:13.444917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26e284-1869-422a-b882-aa6a194c91e5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:13.445478 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26e284-1869-422a-b882-aa6a194c91e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:13.446044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26e284-1869-422a-b882-aa6a194c91e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:13.454433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26e284-1869-422a-b882-aa6a194c91e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:13.456364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26e284-1869-422a-b882-aa6a194c91e5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:13.467244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0326f3-8cd7-4378-addf-d1df5e35ccb2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-fd26e284-1869-422a-b882-aa6a194c91e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:13.476133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:13.476984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:13.477960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:13.477960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:13.477960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:13.477960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:13.477960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:13.477960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:13.477960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:13.477960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:13.480500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 305/608] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:13.486494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26e284-1869-422a-b882-aa6a194c91e5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:13.504036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:13.520458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:13.535653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12d522-dfd0-43d1-8dbf-15668ea494c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:13.536344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12d522-dfd0-43d1-8dbf-15668ea494c5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:13.536621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12d522-dfd0-43d1-8dbf-15668ea494c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:13.536893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12d522-dfd0-43d1-8dbf-15668ea494c5 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:13.537669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12d522-dfd0-43d1-8dbf-15668ea494c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:13.542637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12d522-dfd0-43d1-8dbf-15668ea494c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:13.542637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12d522-dfd0-43d1-8dbf-15668ea494c5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:13.549406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 304/609] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:13.556139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12d522-dfd0-43d1-8dbf-15668ea494c5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:13.659873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26e284-1869-422a-b882-aa6a194c91e5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:13.690981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12d522-dfd0-43d1-8dbf-15668ea494c5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:13.699806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e678e96-f2b1-413c-92c6-109dc5aa4631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:13.700486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e678e96-f2b1-413c-92c6-109dc5aa4631 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:13.700771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e678e96-f2b1-413c-92c6-109dc5aa4631 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:13.701049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e678e96-f2b1-413c-92c6-109dc5aa4631 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:13.701852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e678e96-f2b1-413c-92c6-109dc5aa4631 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:13.702213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e678e96-f2b1-413c-92c6-109dc5aa4631 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:13.703838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e678e96-f2b1-413c-92c6-109dc5aa4631 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:13.718662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 306/610] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:13.724839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e678e96-f2b1-413c-92c6-109dc5aa4631 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:13.735939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdfbaa-fd07-4b2e-a74d-2ce753b7364e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:13.737013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdfbaa-fd07-4b2e-a74d-2ce753b7364e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:13.737507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdfbaa-fd07-4b2e-a74d-2ce753b7364e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:13.737952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdfbaa-fd07-4b2e-a74d-2ce753b7364e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:13.749966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdfbaa-fd07-4b2e-a74d-2ce753b7364e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:13.752984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdfbaa-fd07-4b2e-a74d-2ce753b7364e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8cbdfbaa-fd07-4b2e-a74d-2ce753b7364e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:13.772007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:13.773133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:13.774169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:13.774169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:13.774169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:13.774169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:13.776518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 305/611] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:13.782334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdfbaa-fd07-4b2e-a74d-2ce753b7364e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:13.806826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:13.834284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:13.847445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e678e96-f2b1-413c-92c6-109dc5aa4631 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:13.885399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6e8b4-a7a6-42b7-a1d6-0e4706975a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:13.886438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6e8b4-a7a6-42b7-a1d6-0e4706975a1f None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:13.886634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6e8b4-a7a6-42b7-a1d6-0e4706975a1f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:13.886870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6e8b4-a7a6-42b7-a1d6-0e4706975a1f None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/362797538cbd48dc9c73b09b0548959e/roles/4766b6a3fe4746798f255d0813534a04` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:13.902767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6e8b4-a7a6-42b7-a1d6-0e4706975a1f None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=362797538cbd48dc9c73b09b0548959e, role_id=4766b6a3fe4746798f255d0813534a04)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:13.904220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6e8b4-a7a6-42b7-a1d6-0e4706975a1f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:13.907814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdfbaa-fd07-4b2e-a74d-2ce753b7364e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:13.943191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abe7bc-0aff-44cb-9a87-9ce63b78fb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:13.944812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abe7bc-0aff-44cb-9a87-9ce63b78fb80 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:13.945312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abe7bc-0aff-44cb-9a87-9ce63b78fb80 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:13.945802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abe7bc-0aff-44cb-9a87-9ce63b78fb80 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:13.946929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abe7bc-0aff-44cb-9a87-9ce63b78fb80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:13.947464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abe7bc-0aff-44cb-9a87-9ce63b78fb80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:13.949072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abe7bc-0aff-44cb-9a87-9ce63b78fb80 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:13.962380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 306/612] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:13.969966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abe7bc-0aff-44cb-9a87-9ce63b78fb80 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:13.991884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 307/613] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:38:13 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/362797538cbd48dc9c73b09b0548959e/roles/4766b6a3fe4746798f255d0813534a04 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:38:14.085472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abe7bc-0aff-44cb-9a87-9ce63b78fb80 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:14.088000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:14.125909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042ad7f-5524-4e9b-bde5-b04e019c4cc0 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:14.126479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042ad7f-5524-4e9b-bde5-b04e019c4cc0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:14.126479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042ad7f-5524-4e9b-bde5-b04e019c4cc0 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:14.128178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 308/614] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:14.137044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6c882-5a1d-4481-a0a2-d966af276f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:14.137860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6c882-5a1d-4481-a0a2-d966af276f6c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:14.138171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6c882-5a1d-4481-a0a2-d966af276f6c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:14.138451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6c882-5a1d-4481-a0a2-d966af276f6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:14.143794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03a088-4fe0-4549-aea1-6673128d25ee None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:14.144100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03a088-4fe0-4549-aea1-6673128d25ee None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:14.144354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03a088-4fe0-4549-aea1-6673128d25ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:14.159720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6c882-5a1d-4481-a0a2-d966af276f6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:14.161214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6c882-5a1d-4481-a0a2-d966af276f6c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:14.170542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9a03a088-4fe0-4549-aea1-6673128d25ee None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:14.171747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9a03a088-4fe0-4549-aea1-6673128d25ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ffb6c882-5a1d-4481-a0a2-d966af276f6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:14.191386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:14.192507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:14.194344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 307/615] 10.222.0.42 () {56 vars in 1093 bytes} [Fri Jun 3 09:38:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:14.198761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6c882-5a1d-4481-a0a2-d966af276f6c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:14.282527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 309/616] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:14.304664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03a088-4fe0-4549-aea1-6673128d25ee None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:14.364015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:14.365013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6c882-5a1d-4481-a0a2-d966af276f6c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:14.411581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7e832-9c60-4124-82f3-c23aad4cee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:14.412396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03a088-4fe0-4549-aea1-6673128d25ee None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:14.413034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7e832-9c60-4124-82f3-c23aad4cee8f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:14.413484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7e832-9c60-4124-82f3-c23aad4cee8f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:14.413927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7e832-9c60-4124-82f3-c23aad4cee8f None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:14.415925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7e832-9c60-4124-82f3-c23aad4cee8f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:14.416405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7e832-9c60-4124-82f3-c23aad4cee8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:14.418046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7e832-9c60-4124-82f3-c23aad4cee8f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:14.449254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df84d2-e1e0-4317-87d3-d5cec6b16668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:14.450208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df84d2-e1e0-4317-87d3-d5cec6b16668 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:14.450508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df84d2-e1e0-4317-87d3-d5cec6b16668 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:14.450837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df84d2-e1e0-4317-87d3-d5cec6b16668 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:14.452201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 308/617] 10.222.0.42 () {56 vars in 1133 bytes} [Fri Jun 3 09:38:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:14.457250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7e832-9c60-4124-82f3-c23aad4cee8f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:14.463961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df84d2-e1e0-4317-87d3-d5cec6b16668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-58df84d2-e1e0-4317-87d3-d5cec6b16668 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:14.478428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:14.479374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:14.480293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:14.481686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 310/618] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:14.492257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df84d2-e1e0-4317-87d3-d5cec6b16668 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:14.498518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:14.613303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df84d2-e1e0-4317-87d3-d5cec6b16668 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:14.627922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:14.643835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7e832-9c60-4124-82f3-c23aad4cee8f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:14.649142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e573-9696-4058-8da8-0611df686408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:14.649909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e573-9696-4058-8da8-0611df686408 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:14.650280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e573-9696-4058-8da8-0611df686408 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:14.650629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5e573-9696-4058-8da8-0611df686408 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:14.651492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e573-9696-4058-8da8-0611df686408 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:14.651866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e573-9696-4058-8da8-0611df686408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:14.653316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5e573-9696-4058-8da8-0611df686408 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:14.663195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 311/619] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:14.669349 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e573-9696-4058-8da8-0611df686408 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:14.707272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22fa6a-fd2b-417e-92cb-ed4077375b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:14.707845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22fa6a-fd2b-417e-92cb-ed4077375b57 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:14.708132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22fa6a-fd2b-417e-92cb-ed4077375b57 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:14.708411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22fa6a-fd2b-417e-92cb-ed4077375b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:14.804325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22fa6a-fd2b-417e-92cb-ed4077375b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:14.805817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22fa6a-fd2b-417e-92cb-ed4077375b57 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:14.843656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5e573-9696-4058-8da8-0611df686408 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:14.860455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:14.879871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc28a3c-502c-416b-bdb5-4edb09ed7f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:14.880667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc28a3c-502c-416b-bdb5-4edb09ed7f80 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:14.880950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc28a3c-502c-416b-bdb5-4edb09ed7f80 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:14.881818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc28a3c-502c-416b-bdb5-4edb09ed7f80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:14.892600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc28a3c-502c-416b-bdb5-4edb09ed7f80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:14.894130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc28a3c-502c-416b-bdb5-4edb09ed7f80 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-0bc28a3c-502c-416b-bdb5-4edb09ed7f80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:14.913138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:14.914128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:14.915650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:14.915650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:14.915650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:14.915650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:14.915650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 312/620] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:14.922595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc28a3c-502c-416b-bdb5-4edb09ed7f80 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:14.959905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:15.040317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc28a3c-502c-416b-bdb5-4edb09ed7f80 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:15.054228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 309/621] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 09:38:15.055148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22fa6a-fd2b-417e-92cb-ed4077375b57 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:15.077558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74b435-6367-437d-9be8-d4bb5769de7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:15.078276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74b435-6367-437d-9be8-d4bb5769de7e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:15.078561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74b435-6367-437d-9be8-d4bb5769de7e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:15.079003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74b435-6367-437d-9be8-d4bb5769de7e None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:15.079746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74b435-6367-437d-9be8-d4bb5769de7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:15.080059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74b435-6367-437d-9be8-d4bb5769de7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:15.081440 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b74b435-6367-437d-9be8-d4bb5769de7e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:15.094576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 313/622] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:15.106930 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74b435-6367-437d-9be8-d4bb5769de7e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:15.182139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22fa6a-fd2b-417e-92cb-ed4077375b57 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:15.225284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:15.241410 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74b435-6367-437d-9be8-d4bb5769de7e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:15.248005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:15.263474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdc995-1c89-4440-bab2-812b214290f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:15.264387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdc995-1c89-4440-bab2-812b214290f9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:15.264769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdc995-1c89-4440-bab2-812b214290f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:15.265108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdc995-1c89-4440-bab2-812b214290f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:15.279353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdc995-1c89-4440-bab2-812b214290f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:15.281030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdc995-1c89-4440-bab2-812b214290f9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:15.293607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225342-0049-4ef7-ad5c-a2fcba37c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:15.293607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225342-0049-4ef7-ad5c-a2fcba37c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:15.293607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225342-0049-4ef7-ad5c-a2fcba37c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:15.293607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225342-0049-4ef7-ad5c-a2fcba37c8c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b2bdc995-1c89-4440-bab2-812b214290f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:15.310295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:15.311244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:15.312110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:15.312110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:15.312110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:15.312110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:15.313650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d225342-0049-4ef7-ad5c-a2fcba37c8c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:15.314406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d225342-0049-4ef7-ad5c-a2fcba37c8c2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:15.314847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 310/623] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:15.318892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdc995-1c89-4440-bab2-812b214290f9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-0d225342-0049-4ef7-ad5c-a2fcba37c8c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:15.346711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:15.347743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:15.348691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 314/624] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:15.355432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225342-0049-4ef7-ad5c-a2fcba37c8c2 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:15.480120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225342-0049-4ef7-ad5c-a2fcba37c8c2 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:15.490236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdc995-1c89-4440-bab2-812b214290f9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:15.546852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d826803-04e4-429b-87ad-62981af851b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:15.548279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d826803-04e4-429b-87ad-62981af851b5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:15.548580 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d826803-04e4-429b-87ad-62981af851b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:15.549461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d826803-04e4-429b-87ad-62981af851b5 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:15.550342 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d826803-04e4-429b-87ad-62981af851b5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:15.550658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d826803-04e4-429b-87ad-62981af851b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:15.552107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d826803-04e4-429b-87ad-62981af851b5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:15.557780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac0986-1492-4a23-b75d-4a47b8a20f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:15.558488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac0986-1492-4a23-b75d-4a47b8a20f81 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:15.558773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac0986-1492-4a23-b75d-4a47b8a20f81 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:15.559088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac0986-1492-4a23-b75d-4a47b8a20f81 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:15.559823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac0986-1492-4a23-b75d-4a47b8a20f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:15.560123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac0986-1492-4a23-b75d-4a47b8a20f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:15.561496 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac0986-1492-4a23-b75d-4a47b8a20f81 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:15.591724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 311/625] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:15.592070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 315/626] 10.222.0.42 () {56 vars in 1135 bytes} [Fri Jun 3 09:38:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:15.594760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac0986-1492-4a23-b75d-4a47b8a20f81 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:15.602304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d826803-04e4-429b-87ad-62981af851b5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:15.607171 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:15.660210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:15.775137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac0986-1492-4a23-b75d-4a47b8a20f81 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:15.779662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d826803-04e4-429b-87ad-62981af851b5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:15.819986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9921f-f9a3-4375-842c-a63163511c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:15.820788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9921f-f9a3-4375-842c-a63163511c46 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:15.821066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9921f-f9a3-4375-842c-a63163511c46 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:15.821370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9921f-f9a3-4375-842c-a63163511c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:15.825852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e6a79-772d-472d-b795-8ffca08c3658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:15.826775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e6a79-772d-472d-b795-8ffca08c3658 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:15.827134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e6a79-772d-472d-b795-8ffca08c3658 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:15.827487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e6a79-772d-472d-b795-8ffca08c3658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:15.832051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9921f-f9a3-4375-842c-a63163511c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:15.833534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9921f-f9a3-4375-842c-a63163511c46 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:15.836510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e6a79-772d-472d-b795-8ffca08c3658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:15.838423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e6a79-772d-472d-b795-8ffca08c3658 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3ac9921f-f9a3-4375-842c-a63163511c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:15.853214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:15.854311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:15.855181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:15.855181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:15.855181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:15.855181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6f3e6a79-772d-472d-b795-8ffca08c3658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:15.855406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:15.856195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:15.857695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:15.857695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:15.857695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:15.857695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:15.857695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:15.857695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:15.857695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:15.857695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:15.858153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 312/627] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:15.858153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393b861-63a5-404f-986d-445a616ae6a4 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:15.858153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393b861-63a5-404f-986d-445a616ae6a4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:15.858153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393b861-63a5-404f-986d-445a616ae6a4 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:15.860803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 316/628] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:15.861002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 313/629] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:15.861138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e6a79-772d-472d-b795-8ffca08c3658 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:15.864220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393b861-63a5-404f-986d-445a616ae6a4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:15.933692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:15.966150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:15.973634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e6a79-772d-472d-b795-8ffca08c3658 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:15.983479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393b861-63a5-404f-986d-445a616ae6a4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:16.011416 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30193adc-477a-44f3-829d-1d15a9342a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:16.012586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30193adc-477a-44f3-829d-1d15a9342a0b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.013364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30193adc-477a-44f3-829d-1d15a9342a0b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.014129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30193adc-477a-44f3-829d-1d15a9342a0b None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.015202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30193adc-477a-44f3-829d-1d15a9342a0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:16.015701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30193adc-477a-44f3-829d-1d15a9342a0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:16.017939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30193adc-477a-44f3-829d-1d15a9342a0b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:16.021079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ff2ba-26da-46dc-899e-c531cfdb20c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:16.022380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ff2ba-26da-46dc-899e-c531cfdb20c8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.022734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ff2ba-26da-46dc-899e-c531cfdb20c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.023085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ff2ba-26da-46dc-899e-c531cfdb20c8 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.024013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ff2ba-26da-46dc-899e-c531cfdb20c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:16.024387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ff2ba-26da-46dc-899e-c531cfdb20c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:16.026017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ff2ba-26da-46dc-899e-c531cfdb20c8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:16.034131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 317/630] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:16.036207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be8597-c6c8-4bcb-b4b2-c37fd1ebdd55 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.036997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be8597-c6c8-4bcb-b4b2-c37fd1ebdd55 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.038188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be8597-c6c8-4bcb-b4b2-c37fd1ebdd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.045067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 314/631] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:16.045810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ff2ba-26da-46dc-899e-c531cfdb20c8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:16.057202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-34be8597-c6c8-4bcb-b4b2-c37fd1ebdd55 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:16.059459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-34be8597-c6c8-4bcb-b4b2-c37fd1ebdd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:16.149657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 318/632] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:16.151120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be8597-c6c8-4bcb-b4b2-c37fd1ebdd55 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:16.171006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ff2ba-26da-46dc-899e-c531cfdb20c8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:16.213721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:16.219663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c79483-06f2-46fb-a97f-81d06dd31e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:16.220467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c79483-06f2-46fb-a97f-81d06dd31e45 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.220741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c79483-06f2-46fb-a97f-81d06dd31e45 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.221012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c79483-06f2-46fb-a97f-81d06dd31e45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.230970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c79483-06f2-46fb-a97f-81d06dd31e45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:16.232422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c79483-06f2-46fb-a97f-81d06dd31e45 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-85c79483-06f2-46fb-a97f-81d06dd31e45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:16.246784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:16.252575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 315/633] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:16.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c79483-06f2-46fb-a97f-81d06dd31e45 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:16.260755 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:16.267010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be8597-c6c8-4bcb-b4b2-c37fd1ebdd55 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:16.304853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b74d-68dc-47a6-8495-1275146b4d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:16.305694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b74d-68dc-47a6-8495-1275146b4d43 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.306098 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b74d-68dc-47a6-8495-1275146b4d43 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.306472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b74d-68dc-47a6-8495-1275146b4d43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.307449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20b74d-68dc-47a6-8495-1275146b4d43 None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=dfe8ef12db914a9693ef470f1706bb5a) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:16.307838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20b74d-68dc-47a6-8495-1275146b4d43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:16.309421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20b74d-68dc-47a6-8495-1275146b4d43 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:16.322460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 319/634] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:16 2022] GET /identity/v3/role_assignments?user.id=dfe8ef12db914a9693ef470f1706bb5a&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:16.324376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa590a6-e362-4af5-a529-f49066516e17 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.324901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa590a6-e362-4af5-a529-f49066516e17 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.325359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa590a6-e362-4af5-a529-f49066516e17 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.328433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 320/635] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:16.329521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa590a6-e362-4af5-a529-f49066516e17 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:16.372379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c79483-06f2-46fb-a97f-81d06dd31e45 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:16.414941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89752df0-24e1-4db4-96a6-10a8042f581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:16.415906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89752df0-24e1-4db4-96a6-10a8042f581d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.416259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89752df0-24e1-4db4-96a6-10a8042f581d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.416596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89752df0-24e1-4db4-96a6-10a8042f581d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.428917 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89752df0-24e1-4db4-96a6-10a8042f581d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-89752df0-24e1-4db4-96a6-10a8042f581d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:16.439399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:16.440650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:16.441670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:16.444362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 316/636] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:16.445422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a80ca-58e1-4770-963c-5f56b8d6a2c2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.445811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a80ca-58e1-4770-963c-5f56b8d6a2c2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.446210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a80ca-58e1-4770-963c-5f56b8d6a2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.465408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-018a80ca-58e1-4770-963c-5f56b8d6a2c2 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:16.466846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-018a80ca-58e1-4770-963c-5f56b8d6a2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:16.480330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa590a6-e362-4af5-a529-f49066516e17 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:16.506997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:16.531698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a89e93-2666-40c2-b853-362987bd712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:16.532888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a89e93-2666-40c2-b853-362987bd712f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.533320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a89e93-2666-40c2-b853-362987bd712f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.533767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a89e93-2666-40c2-b853-362987bd712f None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.534758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a89e93-2666-40c2-b853-362987bd712f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:16.535400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a89e93-2666-40c2-b853-362987bd712f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:16.537381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a89e93-2666-40c2-b853-362987bd712f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:16.539950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:16.549965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 317/637] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:16.551084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a80ca-58e1-4770-963c-5f56b8d6a2c2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:16.561473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 321/638] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:16.562787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a89e93-2666-40c2-b853-362987bd712f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:16.674463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a80ca-58e1-4770-963c-5f56b8d6a2c2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:16.689089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a89e93-2666-40c2-b853-362987bd712f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:16.711601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4eb1e2-d96f-4ac1-80a3-8ff4e59d82d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:16.712506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eb1e2-d96f-4ac1-80a3-8ff4e59d82d3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.712794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eb1e2-d96f-4ac1-80a3-8ff4e59d82d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.713079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eb1e2-d96f-4ac1-80a3-8ff4e59d82d3 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.714001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4eb1e2-d96f-4ac1-80a3-8ff4e59d82d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:16.714197 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4eb1e2-d96f-4ac1-80a3-8ff4e59d82d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:16.717030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4eb1e2-d96f-4ac1-80a3-8ff4e59d82d3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:16.724522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 318/639] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:16.725485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4eb1e2-d96f-4ac1-80a3-8ff4e59d82d3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:16.732963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e1a80-05a9-4c71-8f1d-32addac8543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:16.734112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e1a80-05a9-4c71-8f1d-32addac8543f None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.734571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e1a80-05a9-4c71-8f1d-32addac8543f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.735022 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e1a80-05a9-4c71-8f1d-32addac8543f None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/37186dc0010247cd936f160b5eb0298b/roles/4766b6a3fe4746798f255d0813534a04` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.755071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e1a80-05a9-4c71-8f1d-32addac8543f None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=37186dc0010247cd936f160b5eb0298b, role_id=4766b6a3fe4746798f255d0813534a04)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:16.757116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e1a80-05a9-4c71-8f1d-32addac8543f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:16.787333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:16.824369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 322/640] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:38:16 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/37186dc0010247cd936f160b5eb0298b/roles/4766b6a3fe4746798f255d0813534a04 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:38:16.825903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c818ea-83d2-497c-9e5f-e334f8349f30 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.826214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c818ea-83d2-497c-9e5f-e334f8349f30 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.826497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c818ea-83d2-497c-9e5f-e334f8349f30 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.827969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 323/641] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:16.832828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c818ea-83d2-497c-9e5f-e334f8349f30 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:16.836337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4eb1e2-d96f-4ac1-80a3-8ff4e59d82d3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:16.870056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:16.889492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eaf12c-7f8b-4ea6-bcdd-9e03b4cabef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:16.890936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaf12c-7f8b-4ea6-bcdd-9e03b4cabef5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.891489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaf12c-7f8b-4ea6-bcdd-9e03b4cabef5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.892121 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaf12c-7f8b-4ea6-bcdd-9e03b4cabef5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.909933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eaf12c-7f8b-4ea6-bcdd-9e03b4cabef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-46eaf12c-7f8b-4ea6-bcdd-9e03b4cabef5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:16.922338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:16.923351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:16.924214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:16.933259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 319/642] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:16.935084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e94bd-a3f9-452a-8559-a765904c891c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:16.936037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e94bd-a3f9-452a-8559-a765904c891c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:16.936477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e94bd-a3f9-452a-8559-a765904c891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:16.961274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-106e94bd-a3f9-452a-8559-a765904c891c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:16.962534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-106e94bd-a3f9-452a-8559-a765904c891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:16.966734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c818ea-83d2-497c-9e5f-e334f8349f30 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.006872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595a819-4408-48ca-bf31-15c00045cfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.007692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595a819-4408-48ca-bf31-15c00045cfe1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.007974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595a819-4408-48ca-bf31-15c00045cfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.008246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595a819-4408-48ca-bf31-15c00045cfe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.020548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595a819-4408-48ca-bf31-15c00045cfe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:17.022387 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595a819-4408-48ca-bf31-15c00045cfe1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:17.041304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 320/643] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-5595a819-4408-48ca-bf31-15c00045cfe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:17.044188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:17.045093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:17.046018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:17.046018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:17.046018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:17.046018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:17.046284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e94bd-a3f9-452a-8559-a765904c891c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.051230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 324/644] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:17.051707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595a819-4408-48ca-bf31-15c00045cfe1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.114439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:17.148135 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:17.170364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595a819-4408-48ca-bf31-15c00045cfe1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.179223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e94bd-a3f9-452a-8559-a765904c891c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.208181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503fe03-e056-460d-a015-37b3347b13f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.208951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503fe03-e056-460d-a015-37b3347b13f3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.209295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503fe03-e056-460d-a015-37b3347b13f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.209646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503fe03-e056-460d-a015-37b3347b13f3 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.210468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503fe03-e056-460d-a015-37b3347b13f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:17.210843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503fe03-e056-460d-a015-37b3347b13f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:17.212315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503fe03-e056-460d-a015-37b3347b13f3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:17.227035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 325/645] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:17.228046 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503fe03-e056-460d-a015-37b3347b13f3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.229002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888f48d-d696-4fb3-ac9d-4315ab6ddf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.230281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888f48d-d696-4fb3-ac9d-4315ab6ddf6e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.231096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888f48d-d696-4fb3-ac9d-4315ab6ddf6e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.231556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888f48d-d696-4fb3-ac9d-4315ab6ddf6e None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.233464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888f48d-d696-4fb3-ac9d-4315ab6ddf6e None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:17.233990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888f48d-d696-4fb3-ac9d-4315ab6ddf6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:17.236692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888f48d-d696-4fb3-ac9d-4315ab6ddf6e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:17.250173 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 321/646] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:17.251307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888f48d-d696-4fb3-ac9d-4315ab6ddf6e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.336424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503fe03-e056-460d-a015-37b3347b13f3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.377058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c04b0-ed2c-473f-9b5f-6a7434ba4aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.377864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c04b0-ed2c-473f-9b5f-6a7434ba4aaf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.378256 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c04b0-ed2c-473f-9b5f-6a7434ba4aaf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.378606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c04b0-ed2c-473f-9b5f-6a7434ba4aaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.387459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:17.390496 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c04b0-ed2c-473f-9b5f-6a7434ba4aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:17.392906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888f48d-d696-4fb3-ac9d-4315ab6ddf6e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-019c04b0-ed2c-473f-9b5f-6a7434ba4aaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:17.398599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:17.399516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:17.400488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:17.404611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 326/647] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:17.405286 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c04b0-ed2c-473f-9b5f-6a7434ba4aaf None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.407824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:17.445019 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf6624-cd49-4c96-b579-583339207954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.446152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf6624-cd49-4c96-b579-583339207954 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.446622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf6624-cd49-4c96-b579-583339207954 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.447108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf6624-cd49-4c96-b579-583339207954 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.455926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf6624-cd49-4c96-b579-583339207954 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:17.457384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf6624-cd49-4c96-b579-583339207954 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-abdf6624-cd49-4c96-b579-583339207954 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:17.473626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:17.474665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:17.475682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 322/648] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:17.476321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf6624-cd49-4c96-b579-583339207954 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.530205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c04b0-ed2c-473f-9b5f-6a7434ba4aaf None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.574645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e5bc-03de-41ed-8718-50c016e69c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.575599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e5bc-03de-41ed-8718-50c016e69c50 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.575888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e5bc-03de-41ed-8718-50c016e69c50 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.576168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e5bc-03de-41ed-8718-50c016e69c50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.590444 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e5bc-03de-41ed-8718-50c016e69c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:17.591932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e5bc-03de-41ed-8718-50c016e69c50 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:17.592349 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf6624-cd49-4c96-b579-583339207954 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-c114e5bc-03de-41ed-8718-50c016e69c50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:17.614423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:17.615361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:17.616811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:17.616811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:17.616811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:17.616811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:17.616811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 327/649] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:17.617834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e5bc-03de-41ed-8718-50c016e69c50 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.634086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91046ffb-df86-4660-a935-fe1de5eab3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.635029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91046ffb-df86-4660-a935-fe1de5eab3f5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.635642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91046ffb-df86-4660-a935-fe1de5eab3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.636136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91046ffb-df86-4660-a935-fe1de5eab3f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.637231 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91046ffb-df86-4660-a935-fe1de5eab3f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:17.637831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91046ffb-df86-4660-a935-fe1de5eab3f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:17.639460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91046ffb-df86-4660-a935-fe1de5eab3f5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:17.650949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 323/650] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:17.652042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91046ffb-df86-4660-a935-fe1de5eab3f5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.685663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:17.691451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:17.754288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e5bc-03de-41ed-8718-50c016e69c50 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.771879 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91046ffb-df86-4660-a935-fe1de5eab3f5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.792528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a1a3-db1c-4421-809e-e7b35cd28fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.793641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a1a3-db1c-4421-809e-e7b35cd28fb8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.794200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a1a3-db1c-4421-809e-e7b35cd28fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.794744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a1a3-db1c-4421-809e-e7b35cd28fb8 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.795841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140a1a3-db1c-4421-809e-e7b35cd28fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:17.796428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140a1a3-db1c-4421-809e-e7b35cd28fb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:17.798144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140a1a3-db1c-4421-809e-e7b35cd28fb8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:17.812472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f32a4c-41d1-49c4-80f5-56b6c260b13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.813143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f32a4c-41d1-49c4-80f5-56b6c260b13e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.813426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f32a4c-41d1-49c4-80f5-56b6c260b13e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.813704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f32a4c-41d1-49c4-80f5-56b6c260b13e None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.814471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f32a4c-41d1-49c4-80f5-56b6c260b13e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:17.814778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f32a4c-41d1-49c4-80f5-56b6c260b13e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:17.816982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f32a4c-41d1-49c4-80f5-56b6c260b13e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:17.820201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 328/651] 10.222.0.42 () {56 vars in 1137 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:17.821348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a1a3-db1c-4421-809e-e7b35cd28fb8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.831221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 324/652] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:17.832050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f32a4c-41d1-49c4-80f5-56b6c260b13e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:17.932126 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a1a3-db1c-4421-809e-e7b35cd28fb8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.944545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:17.945743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:17.960573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f32a4c-41d1-49c4-80f5-56b6c260b13e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:17.973360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f346ec7-738b-4504-bf05-3236e1c9a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:17.974671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f346ec7-738b-4504-bf05-3236e1c9a1b5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:17.975726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f346ec7-738b-4504-bf05-3236e1c9a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:17.976205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f346ec7-738b-4504-bf05-3236e1c9a1b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:17.987501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f346ec7-738b-4504-bf05-3236e1c9a1b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:17.989287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f346ec7-738b-4504-bf05-3236e1c9a1b5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.001994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3a8a4-8bfb-4da2-b7a7-1235cf44da1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:18.002992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a8a4-8bfb-4da2-b7a7-1235cf44da1c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.003794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a8a4-8bfb-4da2-b7a7-1235cf44da1c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.004072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3a8a4-8bfb-4da2-b7a7-1235cf44da1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2f346ec7-738b-4504-bf05-3236e1c9a1b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:18.011516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:18.012368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:18.013333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:18.013333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:18.013333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:18.013333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:18.015661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 329/653] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:18.016785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f346ec7-738b-4504-bf05-3236e1c9a1b5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.085539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3a8a4-8bfb-4da2-b7a7-1235cf44da1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:18.087392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d3a8a4-8bfb-4da2-b7a7-1235cf44da1c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.131597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f346ec7-738b-4504-bf05-3236e1c9a1b5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:18.167657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3ad70-6b4a-4995-a18e-8d67b413e5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:18.168597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3ad70-6b4a-4995-a18e-8d67b413e5e9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.168737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3ad70-6b4a-4995-a18e-8d67b413e5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.168998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3ad70-6b4a-4995-a18e-8d67b413e5e9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.176570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3ad70-6b4a-4995-a18e-8d67b413e5e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:18.178568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3ad70-6b4a-4995-a18e-8d67b413e5e9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.187380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-29a3ad70-6b4a-4995-a18e-8d67b413e5e9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:18.193800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:18.194753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:18.195678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 330/654] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:18.196280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3ad70-6b4a-4995-a18e-8d67b413e5e9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.201561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 325/655] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:18.202706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3a8a4-8bfb-4da2-b7a7-1235cf44da1c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.222115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:18.302241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3ad70-6b4a-4995-a18e-8d67b413e5e9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:18.312279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3a8a4-8bfb-4da2-b7a7-1235cf44da1c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:18.338501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634b0f2-65af-432d-9f59-ebb18fd38e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:18.339059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634b0f2-65af-432d-9f59-ebb18fd38e1f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.339361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634b0f2-65af-432d-9f59-ebb18fd38e1f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.339616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634b0f2-65af-432d-9f59-ebb18fd38e1f None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.340334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634b0f2-65af-432d-9f59-ebb18fd38e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:18.340710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634b0f2-65af-432d-9f59-ebb18fd38e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:18.342112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634b0f2-65af-432d-9f59-ebb18fd38e1f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.351594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd30bf2-7d88-4654-9314-19595e062ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:18.352637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30bf2-7d88-4654-9314-19595e062ff2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.353128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30bf2-7d88-4654-9314-19595e062ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.353609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd30bf2-7d88-4654-9314-19595e062ff2 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.354118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 331/656] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:18.354761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634b0f2-65af-432d-9f59-ebb18fd38e1f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.355487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd30bf2-7d88-4654-9314-19595e062ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:18.356002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd30bf2-7d88-4654-9314-19595e062ff2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:18.358753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd30bf2-7d88-4654-9314-19595e062ff2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.378584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 326/657] 10.222.0.42 () {56 vars in 1137 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:18.379712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd30bf2-7d88-4654-9314-19595e062ff2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.434375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:18.450603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:18.467139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634b0f2-65af-432d-9f59-ebb18fd38e1f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:18.492159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd30bf2-7d88-4654-9314-19595e062ff2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:18.502250 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8ce13-fb86-42e6-8c7c-9a956d5ab606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:18.503332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8ce13-fb86-42e6-8c7c-9a956d5ab606 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.503458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8ce13-fb86-42e6-8c7c-9a956d5ab606 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.503675 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8ce13-fb86-42e6-8c7c-9a956d5ab606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.514244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8ce13-fb86-42e6-8c7c-9a956d5ab606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:18.515845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8ce13-fb86-42e6-8c7c-9a956d5ab606 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.531226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b04dd-0b96-4e07-8221-623a7b06d0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:18.532362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b04dd-0b96-4e07-8221-623a7b06d0a1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.532981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b04dd-0b96-4e07-8221-623a7b06d0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.533755 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b04dd-0b96-4e07-8221-623a7b06d0a1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-90b8ce13-fb86-42e6-8c7c-9a956d5ab606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:18.534556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:18.535422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:18.536324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:18.536324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:18.536324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:18.536324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:18.536324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 332/658] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:18.537024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8ce13-fb86-42e6-8c7c-9a956d5ab606 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.546118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b04dd-0b96-4e07-8221-623a7b06d0a1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:18.547951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b04dd-0b96-4e07-8221-623a7b06d0a1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c93b04dd-0b96-4e07-8221-623a7b06d0a1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:18.564163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:18.565095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 09:38:18.566107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:18.568749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 327/659] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:18.569738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b04dd-0b96-4e07-8221-623a7b06d0a1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.646106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8ce13-fb86-42e6-8c7c-9a956d5ab606 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:18.677931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b04dd-0b96-4e07-8221-623a7b06d0a1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:18.681760 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cef158-5d31-4467-b7d5-221fe41edfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:18.682442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cef158-5d31-4467-b7d5-221fe41edfa1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.682727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cef158-5d31-4467-b7d5-221fe41edfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.683003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cef158-5d31-4467-b7d5-221fe41edfa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.685686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:18.691758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:18.727531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab04590-ce03-4b49-a811-bb349e6604f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:18.728119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab04590-ce03-4b49-a811-bb349e6604f4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.728404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab04590-ce03-4b49-a811-bb349e6604f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.728678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab04590-ce03-4b49-a811-bb349e6604f4 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.729397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab04590-ce03-4b49-a811-bb349e6604f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:18.729705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab04590-ce03-4b49-a811-bb349e6604f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:18.731122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab04590-ce03-4b49-a811-bb349e6604f4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.744916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 328/660] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:18.745676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab04590-ce03-4b49-a811-bb349e6604f4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.775231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cef158-5d31-4467-b7d5-221fe41edfa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:18.776706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cef158-5d31-4467-b7d5-221fe41edfa1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.852312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab04590-ce03-4b49-a811-bb349e6604f4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:18.886199 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 333/661] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:18.887072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cef158-5d31-4467-b7d5-221fe41edfa1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.887966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56836308-ed31-4696-8223-2a353b8d40d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:18.888792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56836308-ed31-4696-8223-2a353b8d40d8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.889227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56836308-ed31-4696-8223-2a353b8d40d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.889506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56836308-ed31-4696-8223-2a353b8d40d8 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.890392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56836308-ed31-4696-8223-2a353b8d40d8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:18.890701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56836308-ed31-4696-8223-2a353b8d40d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:18.892202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56836308-ed31-4696-8223-2a353b8d40d8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:18.914045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 329/662] 10.222.0.42 () {56 vars in 1139 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:18.915490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81da6b-2d7e-489d-82d8-77ebdb373f98 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:18.915785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81da6b-2d7e-489d-82d8-77ebdb373f98 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:18.916068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81da6b-2d7e-489d-82d8-77ebdb373f98 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:18.921565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 330/663] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:18.921565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81da6b-2d7e-489d-82d8-77ebdb373f98 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:18.925482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:18.950918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:18.997363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cef158-5d31-4467-b7d5-221fe41edfa1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.035086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd8b9f-9d4e-428a-bf8d-53f58df26827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.035884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd8b9f-9d4e-428a-bf8d-53f58df26827 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.036167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd8b9f-9d4e-428a-bf8d-53f58df26827 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.036447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd8b9f-9d4e-428a-bf8d-53f58df26827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.038183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81da6b-2d7e-489d-82d8-77ebdb373f98 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.043989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd8b9f-9d4e-428a-bf8d-53f58df26827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:19.045452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd8b9f-9d4e-428a-bf8d-53f58df26827 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-12fd8b9f-9d4e-428a-bf8d-53f58df26827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:19.060577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:19.061494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:19.062412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:19.062412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:19.062412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:19.062412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:19.062412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:19.062412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:19.062412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:19.062412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:19.062412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 334/664] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:19.062885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd8b9f-9d4e-428a-bf8d-53f58df26827 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:19.079281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c1942-d012-4435-8c56-5ea2b409833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.080077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c1942-d012-4435-8c56-5ea2b409833b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.080363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c1942-d012-4435-8c56-5ea2b409833b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.080637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c1942-d012-4435-8c56-5ea2b409833b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.091064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c1942-d012-4435-8c56-5ea2b409833b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:19.092513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c1942-d012-4435-8c56-5ea2b409833b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-437c1942-d012-4435-8c56-5ea2b409833b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:19.112073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:19.112949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:19.113830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:19.113830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:19.113830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:19.113830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:19.120480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 331/665] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:19.120480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28e821-0c58-43cc-8aeb-f059e8987bbd None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.120480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28e821-0c58-43cc-8aeb-f059e8987bbd None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.120480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28e821-0c58-43cc-8aeb-f059e8987bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.134570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cc28e821-0c58-43cc-8aeb-f059e8987bbd None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:19.135760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cc28e821-0c58-43cc-8aeb-f059e8987bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:19.170462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:19.178110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd8b9f-9d4e-428a-bf8d-53f58df26827 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.207102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 332/666] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:19.208831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28e821-0c58-43cc-8aeb-f059e8987bbd None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:19.225647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc93b93-c2bc-44bf-b1fa-e4cd1b8e2574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.227101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc93b93-c2bc-44bf-b1fa-e4cd1b8e2574 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.227571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc93b93-c2bc-44bf-b1fa-e4cd1b8e2574 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.228061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc93b93-c2bc-44bf-b1fa-e4cd1b8e2574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.230589 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:19.299394 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc93b93-c2bc-44bf-b1fa-e4cd1b8e2574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:19.300840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc93b93-c2bc-44bf-b1fa-e4cd1b8e2574 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:19.320172 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28e821-0c58-43cc-8aeb-f059e8987bbd None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.353722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b55c5-fcd0-4dfa-971c-1a220518aa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.354405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b55c5-fcd0-4dfa-971c-1a220518aa77 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.354688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b55c5-fcd0-4dfa-971c-1a220518aa77 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.354964 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b55c5-fcd0-4dfa-971c-1a220518aa77 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.355743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b55c5-fcd0-4dfa-971c-1a220518aa77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:19.356051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b55c5-fcd0-4dfa-971c-1a220518aa77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:19.357483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b55c5-fcd0-4dfa-971c-1a220518aa77 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:19.368628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 333/667] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:19.370790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b55c5-fcd0-4dfa-971c-1a220518aa77 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:19.390637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:19.397514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 335/668] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:19.398340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc93b93-c2bc-44bf-b1fa-e4cd1b8e2574 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:19.442692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:19.471489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b55c5-fcd0-4dfa-971c-1a220518aa77 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.495707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc93b93-c2bc-44bf-b1fa-e4cd1b8e2574 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.505261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35efc16c-83cb-4468-9972-8dfe311fc93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.505940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efc16c-83cb-4468-9972-8dfe311fc93f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.506244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efc16c-83cb-4468-9972-8dfe311fc93f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.506526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efc16c-83cb-4468-9972-8dfe311fc93f None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.507244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efc16c-83cb-4468-9972-8dfe311fc93f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:19.507553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efc16c-83cb-4468-9972-8dfe311fc93f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:19.508947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35efc16c-83cb-4468-9972-8dfe311fc93f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:19.520508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 334/669] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:19.522441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61332d9a-8892-45dc-ba1e-f8394856cb05 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.522739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61332d9a-8892-45dc-ba1e-f8394856cb05 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.523021 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61332d9a-8892-45dc-ba1e-f8394856cb05 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.524268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 335/670] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:19.524961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61332d9a-8892-45dc-ba1e-f8394856cb05 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:19.532364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08515441-fdc4-462b-b046-b4a6634857bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.533553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08515441-fdc4-462b-b046-b4a6634857bf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.533842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08515441-fdc4-462b-b046-b4a6634857bf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.534147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08515441-fdc4-462b-b046-b4a6634857bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.545867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08515441-fdc4-462b-b046-b4a6634857bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-08515441-fdc4-462b-b046-b4a6634857bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:19.553576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:19.554638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 336/671] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:19.555450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08515441-fdc4-462b-b046-b4a6634857bf None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:19.609372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:19.626510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61332d9a-8892-45dc-ba1e-f8394856cb05 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.654499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:19.655152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08515441-fdc4-462b-b046-b4a6634857bf None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.660126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440053d-9547-4b59-8fb1-4f89ac7d089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.660775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440053d-9547-4b59-8fb1-4f89ac7d089a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.661057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440053d-9547-4b59-8fb1-4f89ac7d089a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.661332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440053d-9547-4b59-8fb1-4f89ac7d089a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.662209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440053d-9547-4b59-8fb1-4f89ac7d089a None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=d59a351971e94aa38a22e8113207e5b6) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:19.662528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440053d-9547-4b59-8fb1-4f89ac7d089a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:19.677399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440053d-9547-4b59-8fb1-4f89ac7d089a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:19.677399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 336/672] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:19 2022] GET /identity/v3/role_assignments?user.id=d59a351971e94aa38a22e8113207e5b6&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:19.677399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440053d-9547-4b59-8fb1-4f89ac7d089a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:19.691728 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaed139-e2c5-42f7-9aa1-2c5a2398bae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.692539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaed139-e2c5-42f7-9aa1-2c5a2398bae8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.692825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaed139-e2c5-42f7-9aa1-2c5a2398bae8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.693226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaed139-e2c5-42f7-9aa1-2c5a2398bae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.706218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaed139-e2c5-42f7-9aa1-2c5a2398bae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:19.707704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaed139-e2c5-42f7-9aa1-2c5a2398bae8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ecaed139-e2c5-42f7-9aa1-2c5a2398bae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:19.731529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:19.732494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:19.733485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:19.733485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:19.733485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:19.733485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:19.733485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 337/673] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:19.734443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038faf-8051-4547-b449-48ccea4a0f72 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.734443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038faf-8051-4547-b449-48ccea4a0f72 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.734713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038faf-8051-4547-b449-48ccea4a0f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.752839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-73038faf-8051-4547-b449-48ccea4a0f72 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:19.754138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-73038faf-8051-4547-b449-48ccea4a0f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:19.797214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440053d-9547-4b59-8fb1-4f89ac7d089a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.831473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 338/674] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:19.832355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038faf-8051-4547-b449-48ccea4a0f72 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:19.847064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8432984-8296-49f3-93e4-968d1933d8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.848490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8432984-8296-49f3-93e4-968d1933d8bc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.848932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8432984-8296-49f3-93e4-968d1933d8bc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.849682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8432984-8296-49f3-93e4-968d1933d8bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.860167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8432984-8296-49f3-93e4-968d1933d8bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:19.861644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8432984-8296-49f3-93e4-968d1933d8bc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:19.873630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d8432984-8296-49f3-93e4-968d1933d8bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:19.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:19.877377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:19.878514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:19.878514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:19.878514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:19.878514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:19.878514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:19.878514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:19.878514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:19.878514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:19.878514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 337/675] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:19.879905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8432984-8296-49f3-93e4-968d1933d8bc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:19.941262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:19.952465 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038faf-8051-4547-b449-48ccea4a0f72 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.988405 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dab84-4945-470f-bd1d-659fbd739db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:19.988759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8432984-8296-49f3-93e4-968d1933d8bc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:19.989075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dab84-4945-470f-bd1d-659fbd739db9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:19.989363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dab84-4945-470f-bd1d-659fbd739db9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:19.989671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dab84-4945-470f-bd1d-659fbd739db9 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:19.990533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dab84-4945-470f-bd1d-659fbd739db9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:19.990847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dab84-4945-470f-bd1d-659fbd739db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:19.992213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dab84-4945-470f-bd1d-659fbd739db9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.000573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 339/676] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:20.001364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dab84-4945-470f-bd1d-659fbd739db9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.031493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89a70a-2578-4fe9-a243-0f84e5c4b169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.032159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89a70a-2578-4fe9-a243-0f84e5c4b169 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.032450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89a70a-2578-4fe9-a243-0f84e5c4b169 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.032736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89a70a-2578-4fe9-a243-0f84e5c4b169 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.033452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89a70a-2578-4fe9-a243-0f84e5c4b169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:20.033760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89a70a-2578-4fe9-a243-0f84e5c4b169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.035162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89a70a-2578-4fe9-a243-0f84e5c4b169 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.047787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 338/677] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:20.048426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89a70a-2578-4fe9-a243-0f84e5c4b169 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.114176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:20.118089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dab84-4945-470f-bd1d-659fbd739db9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.151256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89a70a-2578-4fe9-a243-0f84e5c4b169 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.156191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8c624-cab1-4b20-b1b0-491b2bea3f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.156988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8c624-cab1-4b20-b1b0-491b2bea3f13 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.157274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8c624-cab1-4b20-b1b0-491b2bea3f13 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.157552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8c624-cab1-4b20-b1b0-491b2bea3f13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.165903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:20.168901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b8c624-cab1-4b20-b1b0-491b2bea3f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-48b8c624-cab1-4b20-b1b0-491b2bea3f13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:20.176702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:20.183832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 340/678] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:20.184840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8c624-cab1-4b20-b1b0-491b2bea3f13 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.193970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec79af-0350-4faf-adf6-f14d7ccbee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.194650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec79af-0350-4faf-adf6-f14d7ccbee17 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.194930 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec79af-0350-4faf-adf6-f14d7ccbee17 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.195208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec79af-0350-4faf-adf6-f14d7ccbee17 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.196046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec79af-0350-4faf-adf6-f14d7ccbee17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:20.196369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec79af-0350-4faf-adf6-f14d7ccbee17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.198369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec79af-0350-4faf-adf6-f14d7ccbee17 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.211577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 339/679] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:20.212449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec79af-0350-4faf-adf6-f14d7ccbee17 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.297343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8c624-cab1-4b20-b1b0-491b2bea3f13 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.318079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec79af-0350-4faf-adf6-f14d7ccbee17 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.332947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d3f0-9928-42b8-83dd-aa9bc78e0891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.333731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d3f0-9928-42b8-83dd-aa9bc78e0891 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.334556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d3f0-9928-42b8-83dd-aa9bc78e0891 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.334840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d3f0-9928-42b8-83dd-aa9bc78e0891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.345278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d3f0-9928-42b8-83dd-aa9bc78e0891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.346761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d3f0-9928-42b8-83dd-aa9bc78e0891 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.354171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a20714-e6e7-4d54-9f2e-8613f685ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.355469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a20714-e6e7-4d54-9f2e-8613f685ee32 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.355898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a20714-e6e7-4d54-9f2e-8613f685ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.356315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a20714-e6e7-4d54-9f2e-8613f685ee32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.363005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a20714-e6e7-4d54-9f2e-8613f685ee32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b823d3f0-9928-42b8-83dd-aa9bc78e0891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:20.364069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:20.365125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:20.366173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:20.366173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:20.366173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:20.366173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:20.366173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 341/680] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:20.367150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a20714-e6e7-4d54-9f2e-8613f685ee32 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.370527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d3f0-9928-42b8-83dd-aa9bc78e0891 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-52a20714-e6e7-4d54-9f2e-8613f685ee32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:20.384306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:20.385262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:20.386316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:20.386316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:20.386316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:20.386316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:20.386316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:20.386316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:20.386316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:20.386316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:20.387465 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:20.388302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 340/681] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:20.389357 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a20714-e6e7-4d54-9f2e-8613f685ee32 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.433692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:20.478204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d3f0-9928-42b8-83dd-aa9bc78e0891 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.508741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a20714-e6e7-4d54-9f2e-8613f685ee32 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.514485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a9678-68d7-45b3-adc6-79a3583b2c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.515223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a9678-68d7-45b3-adc6-79a3583b2c7a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.515577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a9678-68d7-45b3-adc6-79a3583b2c7a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.515920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a9678-68d7-45b3-adc6-79a3583b2c7a None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.517148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a9678-68d7-45b3-adc6-79a3583b2c7a None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:20.517526 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a9678-68d7-45b3-adc6-79a3583b2c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.519367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a9678-68d7-45b3-adc6-79a3583b2c7a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.532878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 342/682] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:20.532878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a9678-68d7-45b3-adc6-79a3583b2c7a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.545629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6f03e-69dc-4885-8971-e70ee67d01dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.546394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6f03e-69dc-4885-8971-e70ee67d01dc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.546774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6f03e-69dc-4885-8971-e70ee67d01dc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.547094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6f03e-69dc-4885-8971-e70ee67d01dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.547999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6f03e-69dc-4885-8971-e70ee67d01dc None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=aee48298b3834984a12fe2fe0a6f9a77) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:20.548380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6f03e-69dc-4885-8971-e70ee67d01dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.549949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6f03e-69dc-4885-8971-e70ee67d01dc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.565985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 341/683] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/role_assignments?user.id=aee48298b3834984a12fe2fe0a6f9a77&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:20.565985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6f03e-69dc-4885-8971-e70ee67d01dc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.629039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:20.662093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a9678-68d7-45b3-adc6-79a3583b2c7a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.694636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:20.698871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6f03e-69dc-4885-8971-e70ee67d01dc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.712638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc818-27a2-4850-8233-00dbd9cafea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.713567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc818-27a2-4850-8233-00dbd9cafea7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.714026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc818-27a2-4850-8233-00dbd9cafea7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.714448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc818-27a2-4850-8233-00dbd9cafea7 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.715528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bc818-27a2-4850-8233-00dbd9cafea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:20.715982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bc818-27a2-4850-8233-00dbd9cafea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.718265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bc818-27a2-4850-8233-00dbd9cafea7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.737411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 343/684] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:20.738807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc818-27a2-4850-8233-00dbd9cafea7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.748116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce507d-5f99-4036-a61c-b0ca469eb78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.749143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce507d-5f99-4036-a61c-b0ca469eb78c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.749606 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce507d-5f99-4036-a61c-b0ca469eb78c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.750080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce507d-5f99-4036-a61c-b0ca469eb78c None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.751777 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce507d-5f99-4036-a61c-b0ca469eb78c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:20.752344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce507d-5f99-4036-a61c-b0ca469eb78c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.755018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce507d-5f99-4036-a61c-b0ca469eb78c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.774036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 342/685] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:20.775627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce507d-5f99-4036-a61c-b0ca469eb78c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.864635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc818-27a2-4850-8233-00dbd9cafea7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.895951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce507d-5f99-4036-a61c-b0ca469eb78c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:20.907034 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:20.915003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168a202-f980-45e9-9897-37c21f266701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.915867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168a202-f980-45e9-9897-37c21f266701 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.921533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168a202-f980-45e9-9897-37c21f266701 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.921533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168a202-f980-45e9-9897-37c21f266701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.927219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168a202-f980-45e9-9897-37c21f266701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.928729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168a202-f980-45e9-9897-37c21f266701 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.938648 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563197ef-36ff-4fde-9500-ed829ed9e9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:20.939716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563197ef-36ff-4fde-9500-ed829ed9e9ac None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:20.940178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563197ef-36ff-4fde-9500-ed829ed9e9ac None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:20.940639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563197ef-36ff-4fde-9500-ed829ed9e9ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-c168a202-f980-45e9-9897-37c21f266701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:20.949016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:20.949986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:20.950860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:20.950860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:20.950860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:20.950860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:20.951929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563197ef-36ff-4fde-9500-ed829ed9e9ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:20.953570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563197ef-36ff-4fde-9500-ed829ed9e9ac None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:20.954926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 344/686] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:20.955865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168a202-f980-45e9-9897-37c21f266701 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-563197ef-36ff-4fde-9500-ed829ed9e9ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:20.971016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:20.971962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:20.972892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:20.975605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 343/687] 10.222.0.42 () {56 vars in 1093 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:20.976617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563197ef-36ff-4fde-9500-ed829ed9e9ac None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:20.995754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:21.066246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168a202-f980-45e9-9897-37c21f266701 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:21.081775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563197ef-36ff-4fde-9500-ed829ed9e9ac None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:21.102157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d778a0-b5d3-4523-91c7-9d44b4f66804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:21.102814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d778a0-b5d3-4523-91c7-9d44b4f66804 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:21.103095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d778a0-b5d3-4523-91c7-9d44b4f66804 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:21.103370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d778a0-b5d3-4523-91c7-9d44b4f66804 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:21.104196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d778a0-b5d3-4523-91c7-9d44b4f66804 None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=362797538cbd48dc9c73b09b0548959e) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:21.104504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d778a0-b5d3-4523-91c7-9d44b4f66804 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:21.106134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d778a0-b5d3-4523-91c7-9d44b4f66804 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:21.112716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 345/688] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/role_assignments?user.id=362797538cbd48dc9c73b09b0548959e&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:21.113680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d778a0-b5d3-4523-91c7-9d44b4f66804 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:21.121939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e5877-e68c-45c5-b7bd-6b1cdf7cbd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:21.123009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e5877-e68c-45c5-b7bd-6b1cdf7cbd56 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:21.123601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e5877-e68c-45c5-b7bd-6b1cdf7cbd56 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:21.124079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e5877-e68c-45c5-b7bd-6b1cdf7cbd56 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:21.125029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e5877-e68c-45c5-b7bd-6b1cdf7cbd56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:21.125536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e5877-e68c-45c5-b7bd-6b1cdf7cbd56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:21.127260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e5877-e68c-45c5-b7bd-6b1cdf7cbd56 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:21.141571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 344/689] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:21.155599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e5877-e68c-45c5-b7bd-6b1cdf7cbd56 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:21.186423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:21.238703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d778a0-b5d3-4523-91c7-9d44b4f66804 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:21.268388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:21.289855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5215a-f295-411d-9e28-3d201f242db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:21.290564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5215a-f295-411d-9e28-3d201f242db5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:21.290850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5215a-f295-411d-9e28-3d201f242db5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:21.291130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5215a-f295-411d-9e28-3d201f242db5 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:21.291989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5215a-f295-411d-9e28-3d201f242db5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:21.292304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5215a-f295-411d-9e28-3d201f242db5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:21.293798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5215a-f295-411d-9e28-3d201f242db5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:21.315514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e5877-e68c-45c5-b7bd-6b1cdf7cbd56 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:21.332452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 346/690] 10.222.0.42 () {56 vars in 1133 bytes} [Fri Jun 3 09:38:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:21.338560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5215a-f295-411d-9e28-3d201f242db5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:21.401274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab67fe-5aa9-4aa9-8e8d-6cf33b325c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:21.402111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab67fe-5aa9-4aa9-8e8d-6cf33b325c9e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:21.402398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab67fe-5aa9-4aa9-8e8d-6cf33b325c9e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:21.402680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab67fe-5aa9-4aa9-8e8d-6cf33b325c9e None admin] PATH_INFO: `/v3/users/swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:21.422797 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab67fe-5aa9-4aa9-8e8d-6cf33b325c9e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:21.424194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab67fe-5aa9-4aa9-8e8d-6cf33b325c9e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-75ab67fe-5aa9-4aa9-8e8d-6cf33b325c9e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:21.442501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:21.443514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:21.444511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:21.448485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 345/691] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:21.449719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab67fe-5aa9-4aa9-8e8d-6cf33b325c9e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:21.457143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5215a-f295-411d-9e28-3d201f242db5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:21.465746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:21.501040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e465dbc-178e-4edd-90e5-acb9004018af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:21.501733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e465dbc-178e-4edd-90e5-acb9004018af None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:21.503637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e465dbc-178e-4edd-90e5-acb9004018af None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:21.503959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e465dbc-178e-4edd-90e5-acb9004018af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:21.579670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab67fe-5aa9-4aa9-8e8d-6cf33b325c9e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:21.593725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e465dbc-178e-4edd-90e5-acb9004018af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:21.596091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e465dbc-178e-4edd-90e5-acb9004018af None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:21.617421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2d6c-80bc-472f-a9fd-a198f2434d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:21.618299 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2d6c-80bc-472f-a9fd-a198f2434d25 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:21.618682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2d6c-80bc-472f-a9fd-a198f2434d25 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:21.619059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2d6c-80bc-472f-a9fd-a198f2434d25 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:21.620225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc2d6c-80bc-472f-a9fd-a198f2434d25 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:21.620425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc2d6c-80bc-472f-a9fd-a198f2434d25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:21.622028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc2d6c-80bc-472f-a9fd-a198f2434d25 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:21.632519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:21.648335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 346/692] 10.222.0.42 () {56 vars in 1135 bytes} [Fri Jun 3 09:38:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:21.650483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2d6c-80bc-472f-a9fd-a198f2434d25 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:21.737110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 347/693] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:21.743161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e465dbc-178e-4edd-90e5-acb9004018af None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:21.754340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:21.829110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2d6c-80bc-472f-a9fd-a198f2434d25 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:21.892718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e465dbc-178e-4edd-90e5-acb9004018af None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:21.896053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd109d66-7749-4859-9535-93a0ea804737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:21.897223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd109d66-7749-4859-9535-93a0ea804737 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:21.897503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd109d66-7749-4859-9535-93a0ea804737 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:21.897777 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd109d66-7749-4859-9535-93a0ea804737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:21.975271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:21.976600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd109d66-7749-4859-9535-93a0ea804737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:21.978122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd109d66-7749-4859-9535-93a0ea804737 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:21.999597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f4a77-e38b-4aa4-9e80-10a6c2927b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:22.000402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f4a77-e38b-4aa4-9e80-10a6c2927b83 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.000683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f4a77-e38b-4aa4-9e80-10a6c2927b83 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.000964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f4a77-e38b-4aa4-9e80-10a6c2927b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.020221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f4a77-e38b-4aa4-9e80-10a6c2927b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:22.022318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f4a77-e38b-4aa4-9e80-10a6c2927b83 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2c4f4a77-e38b-4aa4-9e80-10a6c2927b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:22.055730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:22.056673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:22.057541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:22.057541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:22.057541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:22.057541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:22.066360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 348/694] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:22.072257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f4a77-e38b-4aa4-9e80-10a6c2927b83 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:22.116242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 347/695] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 09:38:22.117957 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fc045-29ae-472f-8a37-bf17857b8739 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.118252 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fc045-29ae-472f-8a37-bf17857b8739 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.118864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fc045-29ae-472f-8a37-bf17857b8739 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.120810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 348/696] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:22.126775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fc045-29ae-472f-8a37-bf17857b8739 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:22.163937 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:22.212550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f4a77-e38b-4aa4-9e80-10a6c2927b83 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:22.243899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:22.249927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb8eb2-e842-4844-85c1-d04214cb3956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:22.250895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fc045-29ae-472f-8a37-bf17857b8739 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:22.251166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb8eb2-e842-4844-85c1-d04214cb3956 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.251659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb8eb2-e842-4844-85c1-d04214cb3956 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.252194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb8eb2-e842-4844-85c1-d04214cb3956 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.253191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb8eb2-e842-4844-85c1-d04214cb3956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:22.253701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb8eb2-e842-4844-85c1-d04214cb3956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:22.255279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb8eb2-e842-4844-85c1-d04214cb3956 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:22.273532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 349/697] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:22.275201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a7f49-1dc3-47bc-8411-8b5f2afa8bcb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.275727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a7f49-1dc3-47bc-8411-8b5f2afa8bcb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.276385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a7f49-1dc3-47bc-8411-8b5f2afa8bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.293060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212eea5-3f31-4f89-bf56-6d9adfd9f7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:22.293859 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212eea5-3f31-4f89-bf56-6d9adfd9f7e8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.294171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212eea5-3f31-4f89-bf56-6d9adfd9f7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.294452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212eea5-3f31-4f89-bf56-6d9adfd9f7e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.297138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-906a7f49-1dc3-47bc-8411-8b5f2afa8bcb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:22.298586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-906a7f49-1dc3-47bc-8411-8b5f2afa8bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:22.305898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212eea5-3f31-4f89-bf56-6d9adfd9f7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:22.307786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212eea5-3f31-4f89-bf56-6d9adfd9f7e8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3212eea5-3f31-4f89-bf56-6d9adfd9f7e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:22.329607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:22.330548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:22.331865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:22.331865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:22.331865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:22.331865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:22.333984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 349/698] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:22.335624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212eea5-3f31-4f89-bf56-6d9adfd9f7e8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:22.394029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 350/699] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:22.395584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a7f49-1dc3-47bc-8411-8b5f2afa8bcb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:22.460077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:22.474341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212eea5-3f31-4f89-bf56-6d9adfd9f7e8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:22.516046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14304d-7036-4377-b4be-a4fb7c360e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:22.517045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14304d-7036-4377-b4be-a4fb7c360e62 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.517331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14304d-7036-4377-b4be-a4fb7c360e62 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.517610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14304d-7036-4377-b4be-a4fb7c360e62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.522345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a7f49-1dc3-47bc-8411-8b5f2afa8bcb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:22.530985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14304d-7036-4377-b4be-a4fb7c360e62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:22.533010 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14304d-7036-4377-b4be-a4fb7c360e62 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-9a14304d-7036-4377-b4be-a4fb7c360e62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:22.554674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:22.555625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 350/700] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:22.561486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14304d-7036-4377-b4be-a4fb7c360e62 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:22.569776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e7823-0a77-45bf-8961-b906c57c06e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:22.570730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e7823-0a77-45bf-8961-b906c57c06e9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.571238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e7823-0a77-45bf-8961-b906c57c06e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.571742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e7823-0a77-45bf-8961-b906c57c06e9 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.572674 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e7823-0a77-45bf-8961-b906c57c06e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:22.573184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e7823-0a77-45bf-8961-b906c57c06e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:22.573637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:22.575397 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571e7823-0a77-45bf-8961-b906c57c06e9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:22.587947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 351/701] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:22.589012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e7823-0a77-45bf-8961-b906c57c06e9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:22.679505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14304d-7036-4377-b4be-a4fb7c360e62 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:22.716690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222d97c-7637-4cf8-b74f-63b31346b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:22.717991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222d97c-7637-4cf8-b74f-63b31346b409 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.718538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222d97c-7637-4cf8-b74f-63b31346b409 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.719026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222d97c-7637-4cf8-b74f-63b31346b409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.731225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222d97c-7637-4cf8-b74f-63b31346b409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:22.733657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222d97c-7637-4cf8-b74f-63b31346b409 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:22.734532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e7823-0a77-45bf-8961-b906c57c06e9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:22.746459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-e222d97c-7637-4cf8-b74f-63b31346b409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:22.753338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:22.759594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:22.760472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:22.760472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:22.760472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:22.760472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:22.760472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 351/702] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:22.760472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222d97c-7637-4cf8-b74f-63b31346b409 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:22.782662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61b80-f383-4dde-b160-fbb067db36af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:22.783252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61b80-f383-4dde-b160-fbb067db36af None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.783536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61b80-f383-4dde-b160-fbb067db36af None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.783813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61b80-f383-4dde-b160-fbb067db36af None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.785145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61b80-f383-4dde-b160-fbb067db36af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:22.785466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61b80-f383-4dde-b160-fbb067db36af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:22.787311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61b80-f383-4dde-b160-fbb067db36af None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:22.809707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 352/703] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:22.809707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61b80-f383-4dde-b160-fbb067db36af None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:22.839348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:22.889184 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222d97c-7637-4cf8-b74f-63b31346b409 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:22.931477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bf579-0240-4985-92c8-8271be10337d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:22.932711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bf579-0240-4985-92c8-8271be10337d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.933008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bf579-0240-4985-92c8-8271be10337d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.933288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598bf579-0240-4985-92c8-8271be10337d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.939487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61b80-f383-4dde-b160-fbb067db36af None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:22.941595 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bf579-0240-4985-92c8-8271be10337d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:22.944493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598bf579-0240-4985-92c8-8271be10337d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-598bf579-0240-4985-92c8-8271be10337d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:22.963520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:22.964899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:22.966304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:22.966304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:22.966304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:22.966304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:22.966304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:22.966304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:22.966304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:22.966304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:22.969318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 352/704] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:22.970336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bf579-0240-4985-92c8-8271be10337d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:22.984990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c97cf-cf3c-4f8e-839c-3ba580495eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:22.986128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c97cf-cf3c-4f8e-839c-3ba580495eca None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:22.986747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c97cf-cf3c-4f8e-839c-3ba580495eca None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:22.987247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c97cf-cf3c-4f8e-839c-3ba580495eca None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:22.988297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c97cf-cf3c-4f8e-839c-3ba580495eca None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:22.988811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c97cf-cf3c-4f8e-839c-3ba580495eca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:22.990485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c97cf-cf3c-4f8e-839c-3ba580495eca None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:23.003694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 353/705] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:23.004891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c97cf-cf3c-4f8e-839c-3ba580495eca None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:23.035934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:23.105320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598bf579-0240-4985-92c8-8271be10337d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:23.111032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c97cf-cf3c-4f8e-839c-3ba580495eca None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:23.130493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:23.145223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743624be-44bb-453f-ba5d-a114d1ae4dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:23.146216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743624be-44bb-453f-ba5d-a114d1ae4dc5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.147223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743624be-44bb-453f-ba5d-a114d1ae4dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.147660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743624be-44bb-453f-ba5d-a114d1ae4dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.148767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743624be-44bb-453f-ba5d-a114d1ae4dc5 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=dfe8ef12db914a9693ef470f1706bb5a) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:23.149298 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743624be-44bb-453f-ba5d-a114d1ae4dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:23.151253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743624be-44bb-453f-ba5d-a114d1ae4dc5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:23.153367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d01c5-55c5-499d-8156-7c6a13c0276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:23.154348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d01c5-55c5-499d-8156-7c6a13c0276e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.154632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d01c5-55c5-499d-8156-7c6a13c0276e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.154912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d01c5-55c5-499d-8156-7c6a13c0276e None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.156143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d01c5-55c5-499d-8156-7c6a13c0276e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:23.156464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d01c5-55c5-499d-8156-7c6a13c0276e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:23.158205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d01c5-55c5-499d-8156-7c6a13c0276e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:23.166050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 353/706] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:22 2022] GET /identity/v3/role_assignments?user.id=dfe8ef12db914a9693ef470f1706bb5a&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=fe4d636f5c4842f18569347911e6cdec => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:23.167527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743624be-44bb-453f-ba5d-a114d1ae4dc5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:23.174269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 354/707] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:23.175625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268f3e3-fcef-4128-8d8c-9b80260863fc None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.175916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268f3e3-fcef-4128-8d8c-9b80260863fc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.176193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268f3e3-fcef-4128-8d8c-9b80260863fc None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.177679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 355/708] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:23.183332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caa69c-aa5e-43d1-b379-318fb9c558fc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.185922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caa69c-aa5e-43d1-b379-318fb9c558fc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.185922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42caa69c-aa5e-43d1-b379-318fb9c558fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.205388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-42caa69c-aa5e-43d1-b379-318fb9c558fc None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:23.206909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-42caa69c-aa5e-43d1-b379-318fb9c558fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:23.287550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 356/709] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:23.287550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caa69c-aa5e-43d1-b379-318fb9c558fc None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:23.323422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:23.334743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743624be-44bb-453f-ba5d-a114d1ae4dc5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:23.375653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f59b3-9226-4814-bfe5-6b6fba5a2cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:23.376335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f59b3-9226-4814-bfe5-6b6fba5a2cca None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.376618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f59b3-9226-4814-bfe5-6b6fba5a2cca None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.376894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f59b3-9226-4814-bfe5-6b6fba5a2cca None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.377600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f59b3-9226-4814-bfe5-6b6fba5a2cca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:23.380165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f59b3-9226-4814-bfe5-6b6fba5a2cca None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:23.432980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42caa69c-aa5e-43d1-b379-318fb9c558fc None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:23.468716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c428fd9-d4a7-4cc6-9de5-0308e5eab1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:23.469381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c428fd9-d4a7-4cc6-9de5-0308e5eab1c7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.469654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c428fd9-d4a7-4cc6-9de5-0308e5eab1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.469960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c428fd9-d4a7-4cc6-9de5-0308e5eab1c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.470803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c428fd9-d4a7-4cc6-9de5-0308e5eab1c7 None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=37186dc0010247cd936f160b5eb0298b) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:23.471117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c428fd9-d4a7-4cc6-9de5-0308e5eab1c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:23.472553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c428fd9-d4a7-4cc6-9de5-0308e5eab1c7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:23.480126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 354/710] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:38:23 2022] POST /identity/v3/users => generated 254 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:23.481588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f59b3-9226-4814-bfe5-6b6fba5a2cca None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:23.486909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 357/711] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:23 2022] GET /identity/v3/role_assignments?user.id=37186dc0010247cd936f160b5eb0298b&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:23.535897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:23.625159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f59b3-9226-4814-bfe5-6b6fba5a2cca None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:23.674134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae944e-099d-402b-b227-246a51647b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:23.675363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae944e-099d-402b-b227-246a51647b32 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.675876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae944e-099d-402b-b227-246a51647b32 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.676368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae944e-099d-402b-b227-246a51647b32 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.677252 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae944e-099d-402b-b227-246a51647b32 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:23.679103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae944e-099d-402b-b227-246a51647b32 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d2ae944e-099d-402b-b227-246a51647b32 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:23.706294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:23.707315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 09:38:23.708176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:23.711100 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 355/712] 10.222.0.42 () {56 vars in 1129 bytes} [Fri Jun 3 09:38:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:23.728690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c428fd9-d4a7-4cc6-9de5-0308e5eab1c7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:23.833075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:23.852449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c428fd9-d4a7-4cc6-9de5-0308e5eab1c7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:23.888193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa778b-ff32-4002-83c9-6e23a813eb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:23.888782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa778b-ff32-4002-83c9-6e23a813eb79 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.889064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa778b-ff32-4002-83c9-6e23a813eb79 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.889335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa778b-ff32-4002-83c9-6e23a813eb79 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.890069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa778b-ff32-4002-83c9-6e23a813eb79 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:23.890374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa778b-ff32-4002-83c9-6e23a813eb79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:23.891740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa778b-ff32-4002-83c9-6e23a813eb79 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:23.896707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 358/713] 10.222.0.42 () {56 vars in 1124 bytes} [Fri Jun 3 09:38:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:23.902292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae944e-099d-402b-b227-246a51647b32 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:23.970409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dab9c-9f7a-4c58-9e8b-d64a50489da0 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.971049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dab9c-9f7a-4c58-9e8b-d64a50489da0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.971553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dab9c-9f7a-4c58-9e8b-d64a50489da0 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.973119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 359/714] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:38:23.978981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175ec6-7864-4f08-a54c-f76ccbb21c5e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:23.979294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175ec6-7864-4f08-a54c-f76ccbb21c5e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:23.979573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15175ec6-7864-4f08-a54c-f76ccbb21c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:23.997155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-15175ec6-7864-4f08-a54c-f76ccbb21c5e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:23.998412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-15175ec6-7864-4f08-a54c-f76ccbb21c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:24.013079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae944e-099d-402b-b227-246a51647b32 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:24.015804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:24.053484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd806c-c488-423d-9d8a-e0071cc79488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:24.054404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd806c-c488-423d-9d8a-e0071cc79488 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:24.054690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd806c-c488-423d-9d8a-e0071cc79488 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:24.055277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd806c-c488-423d-9d8a-e0071cc79488 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:24.055995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd806c-c488-423d-9d8a-e0071cc79488 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:24.056307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd806c-c488-423d-9d8a-e0071cc79488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:24.057726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd806c-c488-423d-9d8a-e0071cc79488 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:24.066681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 356/715] 10.222.0.42 () {56 vars in 1124 bytes} [Fri Jun 3 09:38:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:24.071806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 360/716] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:38:24.094123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd806c-c488-423d-9d8a-e0071cc79488 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:24.239066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd806c-c488-423d-9d8a-e0071cc79488 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:24.296029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b42002-a378-4e02-b7f2-146cd898717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:24.296793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b42002-a378-4e02-b7f2-146cd898717b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:24.297072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b42002-a378-4e02-b7f2-146cd898717b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:24.297343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b42002-a378-4e02-b7f2-146cd898717b None admin] PATH_INFO: `/v3/services/network` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:24.298027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b42002-a378-4e02-b7f2-146cd898717b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:24.299489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b42002-a378-4e02-b7f2-146cd898717b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f0b42002-a378-4e02-b7f2-146cd898717b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:24.316468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:24.317476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 09:38:24.318442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:24.321331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 357/717] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:24 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:24.326899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15175ec6-7864-4f08-a54c-f76ccbb21c5e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:24.450336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:24.476010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15175ec6-7864-4f08-a54c-f76ccbb21c5e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:24.523562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addecac9-3b42-487c-a91e-2406d54a0569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:24.524159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addecac9-3b42-487c-a91e-2406d54a0569 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:24.524444 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addecac9-3b42-487c-a91e-2406d54a0569 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:24.524723 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addecac9-3b42-487c-a91e-2406d54a0569 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:24.525440 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addecac9-3b42-487c-a91e-2406d54a0569 None admin] RBAC: Adding query filter params (name=network) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:24.525751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addecac9-3b42-487c-a91e-2406d54a0569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:24.527150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addecac9-3b42-487c-a91e-2406d54a0569 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:24.531917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 361/718] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:24.538041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b42002-a378-4e02-b7f2-146cd898717b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:24.590934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:24.684773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b42002-a378-4e02-b7f2-146cd898717b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:24.742843 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47dcad-4bbe-4fb9-b220-37d6933421f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:24.743857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47dcad-4bbe-4fb9-b220-37d6933421f1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:24.744368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47dcad-4bbe-4fb9-b220-37d6933421f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:24.744865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47dcad-4bbe-4fb9-b220-37d6933421f1 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:24.745786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47dcad-4bbe-4fb9-b220-37d6933421f1 None admin] RBAC: Adding query filter params (type=network) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:24.746319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47dcad-4bbe-4fb9-b220-37d6933421f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:24.747915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47dcad-4bbe-4fb9-b220-37d6933421f1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:24.755181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 358/719] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:25.787246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8500e-1647-465c-b575-99a4890f6135 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:25.787801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8500e-1647-465c-b575-99a4890f6135 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:25.787801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8500e-1647-465c-b575-99a4890f6135 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:25.789720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 362/720] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:25.794661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af0ad8-b836-4d90-a546-a35b171cbb75 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:25.795055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af0ad8-b836-4d90-a546-a35b171cbb75 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:25.795241 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af0ad8-b836-4d90-a546-a35b171cbb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:25.815923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-49af0ad8-b836-4d90-a546-a35b171cbb75 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:25.817096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-49af0ad8-b836-4d90-a546-a35b171cbb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:25.904504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 359/721] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:25.934845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8500e-1647-465c-b575-99a4890f6135 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:26.047643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8500e-1647-465c-b575-99a4890f6135 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:26.083066 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0757f-c9c9-49e2-9fdd-ebc840635bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:26.084343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0757f-c9c9-49e2-9fdd-ebc840635bf6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.084834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0757f-c9c9-49e2-9fdd-ebc840635bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.085315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0757f-c9c9-49e2-9fdd-ebc840635bf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.093560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:26.102195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0757f-c9c9-49e2-9fdd-ebc840635bf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-aae0757f-c9c9-49e2-9fdd-ebc840635bf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:26.110443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:26.118238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:26.119083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 363/722] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:26.119775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af0ad8-b836-4d90-a546-a35b171cbb75 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:26.176724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:26.290051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af0ad8-b836-4d90-a546-a35b171cbb75 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:26.340054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1dcce-adc0-4329-b623-ad8ae53f2187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:26.340651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1dcce-adc0-4329-b623-ad8ae53f2187 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.340932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1dcce-adc0-4329-b623-ad8ae53f2187 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.341205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1dcce-adc0-4329-b623-ad8ae53f2187 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.341989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1dcce-adc0-4329-b623-ad8ae53f2187 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:26.342299 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1dcce-adc0-4329-b623-ad8ae53f2187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:26.343688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1dcce-adc0-4329-b623-ad8ae53f2187 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:26.353867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 360/723] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:26.361693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0757f-c9c9-49e2-9fdd-ebc840635bf6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:26.516716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0757f-c9c9-49e2-9fdd-ebc840635bf6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:26.546319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4215eac-9e98-4126-8f84-3486f8f1a7ce None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.546634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4215eac-9e98-4126-8f84-3486f8f1a7ce None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.546867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4215eac-9e98-4126-8f84-3486f8f1a7ce None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.548612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 361/724] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:26.554439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167955c1-9982-4570-9904-b7689329b014 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.554727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167955c1-9982-4570-9904-b7689329b014 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.555001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167955c1-9982-4570-9904-b7689329b014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.571324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ebbc7-3ca8-4904-96e1-e97ed095e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:26.572115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ebbc7-3ca8-4904-96e1-e97ed095e324 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.572393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ebbc7-3ca8-4904-96e1-e97ed095e324 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.572668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ebbc7-3ca8-4904-96e1-e97ed095e324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.577074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-167955c1-9982-4570-9904-b7689329b014 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:26.578275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-167955c1-9982-4570-9904-b7689329b014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:26.589515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ebbc7-3ca8-4904-96e1-e97ed095e324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:26.591214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ebbc7-3ca8-4904-96e1-e97ed095e324 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9d7ebbc7-3ca8-4904-96e1-e97ed095e324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:26.608880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:26.610880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:26.611980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:26.611980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:26.611980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:26.611980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:26.611980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 364/725] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:26.621643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0ad71-7c44-414f-a9b8-541e443dee16 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.621972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0ad71-7c44-414f-a9b8-541e443dee16 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.622228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0ad71-7c44-414f-a9b8-541e443dee16 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.623777 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 365/726] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:38:26.628585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725638d8-1707-41d2-86f2-c2ad8db516f6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.628871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725638d8-1707-41d2-86f2-c2ad8db516f6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.629150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725638d8-1707-41d2-86f2-c2ad8db516f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.647561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-725638d8-1707-41d2-86f2-c2ad8db516f6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:26.648932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-725638d8-1707-41d2-86f2-c2ad8db516f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:26.669362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 362/727] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:26.670240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167955c1-9982-4570-9904-b7689329b014 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:26.688535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:26.723962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:26.747913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 366/728] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 09:38:26.749043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725638d8-1707-41d2-86f2-c2ad8db516f6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:26.788206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167955c1-9982-4570-9904-b7689329b014 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:26.823745 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f2c66-7b63-4797-b188-9f8e74bb1de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:26.824508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f2c66-7b63-4797-b188-9f8e74bb1de5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.824797 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f2c66-7b63-4797-b188-9f8e74bb1de5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.825076 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f2c66-7b63-4797-b188-9f8e74bb1de5 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.825791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f2c66-7b63-4797-b188-9f8e74bb1de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:26.826127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f2c66-7b63-4797-b188-9f8e74bb1de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:26.827501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f2c66-7b63-4797-b188-9f8e74bb1de5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:26.843200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 363/729] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:26.843200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29844c5-8a62-44c7-88f6-adefac2dd3aa None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.843200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29844c5-8a62-44c7-88f6-adefac2dd3aa None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.843200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29844c5-8a62-44c7-88f6-adefac2dd3aa None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.843200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 364/730] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:26.843599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29844c5-8a62-44c7-88f6-adefac2dd3aa None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:26.863309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725638d8-1707-41d2-86f2-c2ad8db516f6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:26.899421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cbd6c-2669-4fa9-ab59-6451980edf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:26.900217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cbd6c-2669-4fa9-ab59-6451980edf0a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.900503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cbd6c-2669-4fa9-ab59-6451980edf0a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.900782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cbd6c-2669-4fa9-ab59-6451980edf0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:26.911174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cbd6c-2669-4fa9-ab59-6451980edf0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-5d7cbd6c-2669-4fa9-ab59-6451980edf0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:26.919055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:26.919988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:26.920826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:26.925005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 367/731] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:26.925672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cbd6c-2669-4fa9-ab59-6451980edf0a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:26.936617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:26.960904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29844c5-8a62-44c7-88f6-adefac2dd3aa None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:26.968600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:26.995958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88c90e-a1e1-47d5-bb1d-e7cb46fd68ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:26.997003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88c90e-a1e1-47d5-bb1d-e7cb46fd68ab None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:26.997453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88c90e-a1e1-47d5-bb1d-e7cb46fd68ab None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:26.998006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88c90e-a1e1-47d5-bb1d-e7cb46fd68ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.010696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88c90e-a1e1-47d5-bb1d-e7cb46fd68ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-1e88c90e-a1e1-47d5-bb1d-e7cb46fd68ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:27.020759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:27.022528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:27.023583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:27.025990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 365/732] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:27.027116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cbd6c-2669-4fa9-ab59-6451980edf0a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:27.027594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688651c-b779-4764-9911-b6b3a6579387 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.028009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688651c-b779-4764-9911-b6b3a6579387 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.028427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688651c-b779-4764-9911-b6b3a6579387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.045404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0688651c-b779-4764-9911-b6b3a6579387 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:27.046898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0688651c-b779-4764-9911-b6b3a6579387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:27.063690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1a886-c57a-4a49-91c8-de431b9b3a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:27.064463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a886-c57a-4a49-91c8-de431b9b3a55 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.064748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a886-c57a-4a49-91c8-de431b9b3a55 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.065027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1a886-c57a-4a49-91c8-de431b9b3a55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.073111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1a886-c57a-4a49-91c8-de431b9b3a55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:27.074587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1a886-c57a-4a49-91c8-de431b9b3a55 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-81f1a886-c57a-4a49-91c8-de431b9b3a55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:27.088637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:27.089563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:27.090566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 368/733] 10.222.0.42 () {56 vars in 1093 bytes} [Fri Jun 3 09:38:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:27.091658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1a886-c57a-4a49-91c8-de431b9b3a55 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.122270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 366/734] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:27.126997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688651c-b779-4764-9911-b6b3a6579387 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.199704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1a886-c57a-4a49-91c8-de431b9b3a55 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:27.201170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:27.201309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:27.234070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688651c-b779-4764-9911-b6b3a6579387 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:27.235078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed6dd8-c6c5-48c6-b376-e38e04b935a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:27.235742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6dd8-c6c5-48c6-b376-e38e04b935a7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.236022 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6dd8-c6c5-48c6-b376-e38e04b935a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.236943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6dd8-c6c5-48c6-b376-e38e04b935a7 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.237094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed6dd8-c6c5-48c6-b376-e38e04b935a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:27.237346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed6dd8-c6c5-48c6-b376-e38e04b935a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:27.238688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed6dd8-c6c5-48c6-b376-e38e04b935a7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:27.247817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 369/735] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:27.252765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed6dd8-c6c5-48c6-b376-e38e04b935a7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.272181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd5fd4-4b29-4cdd-ba0c-fa930536f55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:27.272855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd5fd4-4b29-4cdd-ba0c-fa930536f55b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.273138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd5fd4-4b29-4cdd-ba0c-fa930536f55b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.273415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd5fd4-4b29-4cdd-ba0c-fa930536f55b None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.274184 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd5fd4-4b29-4cdd-ba0c-fa930536f55b None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:27.274493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd5fd4-4b29-4cdd-ba0c-fa930536f55b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:27.275874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd5fd4-4b29-4cdd-ba0c-fa930536f55b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:27.283801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 367/736] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:27.284568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd5fd4-4b29-4cdd-ba0c-fa930536f55b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.362348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed6dd8-c6c5-48c6-b376-e38e04b935a7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:27.395520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd5fd4-4b29-4cdd-ba0c-fa930536f55b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:27.398372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d0763-f21b-4f09-a0c1-bd5d866a1817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:27.399048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d0763-f21b-4f09-a0c1-bd5d866a1817 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.399331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d0763-f21b-4f09-a0c1-bd5d866a1817 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.399605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d0763-f21b-4f09-a0c1-bd5d866a1817 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.400448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d0763-f21b-4f09-a0c1-bd5d866a1817 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:27.400759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d0763-f21b-4f09-a0c1-bd5d866a1817 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:27.402207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204d0763-f21b-4f09-a0c1-bd5d866a1817 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:27.420521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 370/737] 10.222.0.42 () {56 vars in 1133 bytes} [Fri Jun 3 09:38:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:27.421427 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d0763-f21b-4f09-a0c1-bd5d866a1817 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.431194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:27.437091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b895a8d-31be-4e8e-a805-013c4136ec90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:27.437795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b895a8d-31be-4e8e-a805-013c4136ec90 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.438113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b895a8d-31be-4e8e-a805-013c4136ec90 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.438398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b895a8d-31be-4e8e-a805-013c4136ec90 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.439112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b895a8d-31be-4e8e-a805-013c4136ec90 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:27.442971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b895a8d-31be-4e8e-a805-013c4136ec90 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:27.447671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:27.509921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 368/738] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:38:27 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:27.509921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b895a8d-31be-4e8e-a805-013c4136ec90 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.538759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d0763-f21b-4f09-a0c1-bd5d866a1817 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:27.609141 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca54a88-dc29-4ea4-9bc2-fdb1325891d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:27.609969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca54a88-dc29-4ea4-9bc2-fdb1325891d5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.610524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca54a88-dc29-4ea4-9bc2-fdb1325891d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.610809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca54a88-dc29-4ea4-9bc2-fdb1325891d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.627572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca54a88-dc29-4ea4-9bc2-fdb1325891d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:27.629367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca54a88-dc29-4ea4-9bc2-fdb1325891d5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:27.634628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b895a8d-31be-4e8e-a805-013c4136ec90 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-eca54a88-dc29-4ea4-9bc2-fdb1325891d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:27.650500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:27.651421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:27.652291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:27.652291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:27.652291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:27.652291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:27.652291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 371/739] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:27.653020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca54a88-dc29-4ea4-9bc2-fdb1325891d5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.683416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f64d16-19d7-4ea6-8c0c-73c533aa3e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:27.684484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f64d16-19d7-4ea6-8c0c-73c533aa3e3a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.684766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f64d16-19d7-4ea6-8c0c-73c533aa3e3a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.685041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f64d16-19d7-4ea6-8c0c-73c533aa3e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.700445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f64d16-19d7-4ea6-8c0c-73c533aa3e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:27.702468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f64d16-19d7-4ea6-8c0c-73c533aa3e3a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f7f64d16-19d7-4ea6-8c0c-73c533aa3e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:27.727327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:27.736103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:27.738267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:27.738267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:27.738267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:27.738267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:27.738267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 369/740] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:27.738267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f64d16-19d7-4ea6-8c0c-73c533aa3e3a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.739061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:27.774183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca54a88-dc29-4ea4-9bc2-fdb1325891d5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:27.782118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:27.825386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5851f59-5a99-45a1-b1d2-96e5693a99cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:27.826261 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5851f59-5a99-45a1-b1d2-96e5693a99cd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.826546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5851f59-5a99-45a1-b1d2-96e5693a99cd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.826982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5851f59-5a99-45a1-b1d2-96e5693a99cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.842376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5851f59-5a99-45a1-b1d2-96e5693a99cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:27.844617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5851f59-5a99-45a1-b1d2-96e5693a99cd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:27.846702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f64d16-19d7-4ea6-8c0c-73c533aa3e3a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d5851f59-5a99-45a1-b1d2-96e5693a99cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:27.866385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:27.867787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:27.868645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:27.868645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:27.868645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:27.868645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:27.868645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 372/741] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:27.868925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5851f59-5a99-45a1-b1d2-96e5693a99cd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.889312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64fd00-6441-4ce8-b648-c7a5cb9b4d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:27.890016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64fd00-6441-4ce8-b648-c7a5cb9b4d29 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.890302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64fd00-6441-4ce8-b648-c7a5cb9b4d29 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.890578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64fd00-6441-4ce8-b648-c7a5cb9b4d29 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.891298 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64fd00-6441-4ce8-b648-c7a5cb9b4d29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:27.891603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64fd00-6441-4ce8-b648-c7a5cb9b4d29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:27.892978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64fd00-6441-4ce8-b648-c7a5cb9b4d29 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:27.909094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 370/742] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:27.909094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b4488-8acd-43fd-b341-d089eb8323b6 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:27.909094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b4488-8acd-43fd-b341-d089eb8323b6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:27.909094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b4488-8acd-43fd-b341-d089eb8323b6 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:27.909094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 371/743] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:27.909094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b4488-8acd-43fd-b341-d089eb8323b6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:27.987743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5851f59-5a99-45a1-b1d2-96e5693a99cd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.010988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:28.020751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b4488-8acd-43fd-b341-d089eb8323b6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.022550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:28.023421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465bb98-29e2-46de-8eff-7830e8e40028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.024286 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465bb98-29e2-46de-8eff-7830e8e40028 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.024391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465bb98-29e2-46de-8eff-7830e8e40028 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.024769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465bb98-29e2-46de-8eff-7830e8e40028 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.025523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465bb98-29e2-46de-8eff-7830e8e40028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:28.025833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465bb98-29e2-46de-8eff-7830e8e40028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:28.027231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465bb98-29e2-46de-8eff-7830e8e40028 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:28.040018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 373/744] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:28.040835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465bb98-29e2-46de-8eff-7830e8e40028 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:28.059193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfb8e0-e3ea-4607-b52f-6a767b874d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.059889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb8e0-e3ea-4607-b52f-6a767b874d98 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.060172 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb8e0-e3ea-4607-b52f-6a767b874d98 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.060446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb8e0-e3ea-4607-b52f-6a767b874d98 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.061161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfb8e0-e3ea-4607-b52f-6a767b874d98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:28.061465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfb8e0-e3ea-4607-b52f-6a767b874d98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:28.062875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfb8e0-e3ea-4607-b52f-6a767b874d98 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:28.074050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 372/745] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:28.075372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526011fa-f1b4-4db1-9936-30f728e49cf5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.075794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526011fa-f1b4-4db1-9936-30f728e49cf5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.076006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526011fa-f1b4-4db1-9936-30f728e49cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.094479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-526011fa-f1b4-4db1-9936-30f728e49cf5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:28.095682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-526011fa-f1b4-4db1-9936-30f728e49cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:28.154314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465bb98-29e2-46de-8eff-7830e8e40028 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.166813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 373/746] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 09:38:28.167579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526011fa-f1b4-4db1-9936-30f728e49cf5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:28.204198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee0b17-2fe2-471f-9703-a5e9907a9b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.205458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee0b17-2fe2-471f-9703-a5e9907a9b74 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.205925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee0b17-2fe2-471f-9703-a5e9907a9b74 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.206192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee0b17-2fe2-471f-9703-a5e9907a9b74 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.216096 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee0b17-2fe2-471f-9703-a5e9907a9b74 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:28.217519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee0b17-2fe2-471f-9703-a5e9907a9b74 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-95ee0b17-2fe2-471f-9703-a5e9907a9b74 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:28.232143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:28.233288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 374/747] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:28.234683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee0b17-2fe2-471f-9703-a5e9907a9b74 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:28.271183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:28.280269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:28.280507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526011fa-f1b4-4db1-9936-30f728e49cf5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.315528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8d035-1079-4cd6-8641-d6c57d906d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.316332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8d035-1079-4cd6-8641-d6c57d906d05 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.316613 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8d035-1079-4cd6-8641-d6c57d906d05 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.316883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8d035-1079-4cd6-8641-d6c57d906d05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.324644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8d035-1079-4cd6-8641-d6c57d906d05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:28.326137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8d035-1079-4cd6-8641-d6c57d906d05 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:28.341568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee0b17-2fe2-471f-9703-a5e9907a9b74 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-b6f8d035-1079-4cd6-8641-d6c57d906d05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:28.341986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:28.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:28.343818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 374/748] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:38:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:28.344855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8d035-1079-4cd6-8641-d6c57d906d05 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:28.384695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59eea9-12a7-4b0f-a4ec-9028812ecbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.385493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59eea9-12a7-4b0f-a4ec-9028812ecbc6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.385773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59eea9-12a7-4b0f-a4ec-9028812ecbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.386072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59eea9-12a7-4b0f-a4ec-9028812ecbc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.392856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59eea9-12a7-4b0f-a4ec-9028812ecbc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:28.394323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59eea9-12a7-4b0f-a4ec-9028812ecbc6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9a59eea9-12a7-4b0f-a4ec-9028812ecbc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:28.408456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:28.409552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:28.410532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:28.410532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:28.410532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:28.410532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:28.410532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:28.410532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:28.410532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:28.410532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:28.414271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 375/749] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:28.415331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59eea9-12a7-4b0f-a4ec-9028812ecbc6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:28.452913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8d035-1079-4cd6-8641-d6c57d906d05 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.494320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f707c95-724a-4d86-b18b-c2514615d818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.494986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f707c95-724a-4d86-b18b-c2514615d818 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.495270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f707c95-724a-4d86-b18b-c2514615d818 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.495650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f707c95-724a-4d86-b18b-c2514615d818 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.496379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f707c95-724a-4d86-b18b-c2514615d818 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:28.497750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f707c95-724a-4d86-b18b-c2514615d818 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:28.529967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59eea9-12a7-4b0f-a4ec-9028812ecbc6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.531469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:28.557388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 375/750] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:38:28 2022] POST /identity/v3/services => generated 240 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:28.558456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f707c95-724a-4d86-b18b-c2514615d818 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:28.575107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d895534-fb83-45ff-bfc7-145e8a9c7afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.576118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d895534-fb83-45ff-bfc7-145e8a9c7afd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.576632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d895534-fb83-45ff-bfc7-145e8a9c7afd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.577183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d895534-fb83-45ff-bfc7-145e8a9c7afd None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.578244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d895534-fb83-45ff-bfc7-145e8a9c7afd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:28.578766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d895534-fb83-45ff-bfc7-145e8a9c7afd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:28.580628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d895534-fb83-45ff-bfc7-145e8a9c7afd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:28.584413 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:28.602497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 376/751] 10.222.0.42 () {56 vars in 1135 bytes} [Fri Jun 3 09:38:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:28.603655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d895534-fb83-45ff-bfc7-145e8a9c7afd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:28.700758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f707c95-724a-4d86-b18b-c2514615d818 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.749064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d895534-fb83-45ff-bfc7-145e8a9c7afd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.754238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b68b04-e951-40a2-8bae-3c66c49e9dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.754948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b68b04-e951-40a2-8bae-3c66c49e9dde None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.755220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b68b04-e951-40a2-8bae-3c66c49e9dde None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.755495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b68b04-e951-40a2-8bae-3c66c49e9dde None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.756901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b68b04-e951-40a2-8bae-3c66c49e9dde None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:28.757209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b68b04-e951-40a2-8bae-3c66c49e9dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:28.758690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b68b04-e951-40a2-8bae-3c66c49e9dde None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:28.784837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 376/752] 10.222.0.42 () {56 vars in 1149 bytes} [Fri Jun 3 09:38:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:28.784837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b68b04-e951-40a2-8bae-3c66c49e9dde None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:28.791807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef287e6-cb2d-4553-8a8a-95f9affe0f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.792486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef287e6-cb2d-4553-8a8a-95f9affe0f2d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.792774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef287e6-cb2d-4553-8a8a-95f9affe0f2d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.793055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef287e6-cb2d-4553-8a8a-95f9affe0f2d None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.793848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef287e6-cb2d-4553-8a8a-95f9affe0f2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:28.795003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef287e6-cb2d-4553-8a8a-95f9affe0f2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:28.796470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef287e6-cb2d-4553-8a8a-95f9affe0f2d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:28.810918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 377/753] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:28.811840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef287e6-cb2d-4553-8a8a-95f9affe0f2d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:28.838019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:28.872152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:28.911441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b68b04-e951-40a2-8bae-3c66c49e9dde None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.947174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef287e6-cb2d-4553-8a8a-95f9affe0f2d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:28.954563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4997c94-b851-4dce-9d76-f9bcb267f16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.955920 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4997c94-b851-4dce-9d76-f9bcb267f16e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.956424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4997c94-b851-4dce-9d76-f9bcb267f16e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.956899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4997c94-b851-4dce-9d76-f9bcb267f16e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:28.989030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20833560-de70-4604-aa3c-f0bc93dbc281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:28.989618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20833560-de70-4604-aa3c-f0bc93dbc281 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:28.990275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20833560-de70-4604-aa3c-f0bc93dbc281 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:28.990563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20833560-de70-4604-aa3c-f0bc93dbc281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:29.040821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4997c94-b851-4dce-9d76-f9bcb267f16e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:29.042698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4997c94-b851-4dce-9d76-f9bcb267f16e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:29.082330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20833560-de70-4604-aa3c-f0bc93dbc281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:29.083955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20833560-de70-4604-aa3c-f0bc93dbc281 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:29.103126 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:29.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:29.156231 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 377/754] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:38:29.157124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4997c94-b851-4dce-9d76-f9bcb267f16e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:29.203443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 378/755] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:38:29.204386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20833560-de70-4604-aa3c-f0bc93dbc281 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:29.280097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4997c94-b851-4dce-9d76-f9bcb267f16e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:29.316373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24456974-84af-4e36-b189-ac84dd6e8c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:29.317269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24456974-84af-4e36-b189-ac84dd6e8c38 None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:29.317555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24456974-84af-4e36-b189-ac84dd6e8c38 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:29.317835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24456974-84af-4e36-b189-ac84dd6e8c38 None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/dfe8ef12db914a9693ef470f1706bb5a/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:29.320402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24456974-84af-4e36-b189-ac84dd6e8c38 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=dfe8ef12db914a9693ef470f1706bb5a, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:29.321732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20833560-de70-4604-aa3c-f0bc93dbc281 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:29.321952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24456974-84af-4e36-b189-ac84dd6e8c38 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:29.338775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:29.356126 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d90ed-5851-4044-8f9f-0d2df530295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:29.357028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d90ed-5851-4044-8f9f-0d2df530295f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:29.357376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d90ed-5851-4044-8f9f-0d2df530295f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:29.357717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d90ed-5851-4044-8f9f-0d2df530295f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:29.368925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d90ed-5851-4044-8f9f-0d2df530295f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:29.370408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d90ed-5851-4044-8f9f-0d2df530295f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-de9d90ed-5851-4044-8f9f-0d2df530295f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:29.388404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:29.389429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:29.390704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:29.390704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:29.390704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:29.390704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:29.394177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 379/756] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:29.394808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d90ed-5851-4044-8f9f-0d2df530295f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:29.410647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 378/757] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:38:29 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/dfe8ef12db914a9693ef470f1706bb5a/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:38:29.412213 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24456974-84af-4e36-b189-ac84dd6e8c38 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:29.457586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:29.598858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24456974-84af-4e36-b189-ac84dd6e8c38 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:29.616807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d90ed-5851-4044-8f9f-0d2df530295f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:29.645220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5790c0-0a30-4973-b04b-796ea270f544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:29.647946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5790c0-0a30-4973-b04b-796ea270f544 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:29.648465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5790c0-0a30-4973-b04b-796ea270f544 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:29.650667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5790c0-0a30-4973-b04b-796ea270f544 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:29.651676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5790c0-0a30-4973-b04b-796ea270f544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:29.653688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5790c0-0a30-4973-b04b-796ea270f544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:29.655358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5790c0-0a30-4973-b04b-796ea270f544 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:29.684839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 379/758] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:29.691409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5790c0-0a30-4973-b04b-796ea270f544 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:29.696808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739be3ab-8f14-494c-bf93-913c4411dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:29.697384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739be3ab-8f14-494c-bf93-913c4411dc75 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:29.697661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739be3ab-8f14-494c-bf93-913c4411dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:29.697966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739be3ab-8f14-494c-bf93-913c4411dc75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:29.722345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739be3ab-8f14-494c-bf93-913c4411dc75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:29.723812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739be3ab-8f14-494c-bf93-913c4411dc75 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-739be3ab-8f14-494c-bf93-913c4411dc75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:29.743893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:29.744796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:29.745659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:29.745659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:29.745659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:29.745659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:29.745659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 380/759] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:29.754038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739be3ab-8f14-494c-bf93-913c4411dc75 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:29.794426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:29.849824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:29.879853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5790c0-0a30-4973-b04b-796ea270f544 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:29.900717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739be3ab-8f14-494c-bf93-913c4411dc75 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:29.940651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d24f1-2f86-4de0-84b3-b94d75502af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:29.941186 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa81c8-e6ef-4d9b-b1f8-e0e54aa3f953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:29.941458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d24f1-2f86-4de0-84b3-b94d75502af7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:29.941801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d24f1-2f86-4de0-84b3-b94d75502af7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:29.941935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa81c8-e6ef-4d9b-b1f8-e0e54aa3f953 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:29.942129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d24f1-2f86-4de0-84b3-b94d75502af7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:29.942227 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa81c8-e6ef-4d9b-b1f8-e0e54aa3f953 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:29.942416 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa81c8-e6ef-4d9b-b1f8-e0e54aa3f953 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:29.943872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa81c8-e6ef-4d9b-b1f8-e0e54aa3f953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:29.944180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa81c8-e6ef-4d9b-b1f8-e0e54aa3f953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:29.946163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa81c8-e6ef-4d9b-b1f8-e0e54aa3f953 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:29.949698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d24f1-2f86-4de0-84b3-b94d75502af7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:29.951301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d24f1-2f86-4de0-84b3-b94d75502af7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:29.963843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 381/760] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:29.965217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068e4cc-8af9-4e3d-8a1e-7e11d08d987c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:29.965513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068e4cc-8af9-4e3d-8a1e-7e11d08d987c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:29.965791 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068e4cc-8af9-4e3d-8a1e-7e11d08d987c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-025d24f1-2f86-4de0-84b3-b94d75502af7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:29.965909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:29.966690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:29.968190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:29.968190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:29.968190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:29.968190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:29.968190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:29.968190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:29.968190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:29.968190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:29.968609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 382/761] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:29.968609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068e4cc-8af9-4e3d-8a1e-7e11d08d987c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:29.974796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 380/762] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:29.975791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d24f1-2f86-4de0-84b3-b94d75502af7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:30.059342 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:30.082947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068e4cc-8af9-4e3d-8a1e-7e11d08d987c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:30.104283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d24f1-2f86-4de0-84b3-b94d75502af7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:30.119623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bdd78-f8e1-4e21-bfdf-c56d02a74906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:30.120546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bdd78-f8e1-4e21-bfdf-c56d02a74906 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.120976 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bdd78-f8e1-4e21-bfdf-c56d02a74906 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.121403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bdd78-f8e1-4e21-bfdf-c56d02a74906 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.128582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bdd78-f8e1-4e21-bfdf-c56d02a74906 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:30.130280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bdd78-f8e1-4e21-bfdf-c56d02a74906 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-0f2bdd78-f8e1-4e21-bfdf-c56d02a74906 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:30.144766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:30.145824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:30.147290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:30.147290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:30.147290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:30.147290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:30.147290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:30.147290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:30.147290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:30.147290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:30.147704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f72d9-b8bd-4241-84c4-a41ecce3b6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:30.147704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f72d9-b8bd-4241-84c4-a41ecce3b6d3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.147704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f72d9-b8bd-4241-84c4-a41ecce3b6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.153905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f72d9-b8bd-4241-84c4-a41ecce3b6d3 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.154035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 383/763] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:30.154125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f72d9-b8bd-4241-84c4-a41ecce3b6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:30.154125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f72d9-b8bd-4241-84c4-a41ecce3b6d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:30.154297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29139e-52ad-4f73-9715-06daa2f40cfa None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.154297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29139e-52ad-4f73-9715-06daa2f40cfa None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.154297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29139e-52ad-4f73-9715-06daa2f40cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.154566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f72d9-b8bd-4241-84c4-a41ecce3b6d3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:30.168961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 381/764] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:30.169790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f72d9-b8bd-4241-84c4-a41ecce3b6d3 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:30.181849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ba29139e-52ad-4f73-9715-06daa2f40cfa None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:30.183151 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ba29139e-52ad-4f73-9715-06daa2f40cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:30.190205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:30.273131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 384/765] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:30.274172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29139e-52ad-4f73-9715-06daa2f40cfa None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:30.292535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f72d9-b8bd-4241-84c4-a41ecce3b6d3 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:30.335318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ccb7f-a4ea-4bb9-b489-b6ccf8cad4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:30.336440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ccb7f-a4ea-4bb9-b489-b6ccf8cad4df None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.337294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ccb7f-a4ea-4bb9-b489-b6ccf8cad4df None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.337751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ccb7f-a4ea-4bb9-b489-b6ccf8cad4df None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.338740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ccb7f-a4ea-4bb9-b489-b6ccf8cad4df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:30.339239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ccb7f-a4ea-4bb9-b489-b6ccf8cad4df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:30.340850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ccb7f-a4ea-4bb9-b489-b6ccf8cad4df None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:30.354954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 382/766] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:30.356022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ccb7f-a4ea-4bb9-b489-b6ccf8cad4df None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:30.361191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:30.392382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29139e-52ad-4f73-9715-06daa2f40cfa None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:30.428980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d317792-1547-44d7-97d0-09b4d747839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:30.429688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d317792-1547-44d7-97d0-09b4d747839a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.430074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d317792-1547-44d7-97d0-09b4d747839a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.430424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d317792-1547-44d7-97d0-09b4d747839a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.431384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d317792-1547-44d7-97d0-09b4d747839a None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=37186dc0010247cd936f160b5eb0298b) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:30.431749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d317792-1547-44d7-97d0-09b4d747839a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:30.433184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d317792-1547-44d7-97d0-09b4d747839a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:30.438111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:30.444852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 385/767] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:30 2022] GET /identity/v3/role_assignments?user.id=37186dc0010247cd936f160b5eb0298b&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=fe4d636f5c4842f18569347911e6cdec => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:30.445765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d317792-1547-44d7-97d0-09b4d747839a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:30.477257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ccb7f-a4ea-4bb9-b489-b6ccf8cad4df None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:30.518478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364090c-1c6b-4301-bc41-c964a46752ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:30.519708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364090c-1c6b-4301-bc41-c964a46752ef None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.520191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364090c-1c6b-4301-bc41-c964a46752ef None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.520666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364090c-1c6b-4301-bc41-c964a46752ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.521809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364090c-1c6b-4301-bc41-c964a46752ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:30.523592 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364090c-1c6b-4301-bc41-c964a46752ef None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d364090c-1c6b-4301-bc41-c964a46752ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:30.537826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:30.538806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 09:38:30.539874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:30.542934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 383/768] 10.222.0.42 () {56 vars in 1129 bytes} [Fri Jun 3 09:38:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:30.562187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364090c-1c6b-4301-bc41-c964a46752ef None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:30.602272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d317792-1547-44d7-97d0-09b4d747839a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:30.640140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb249a0-6e53-46f0-9d42-bd566c77496b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:30.640729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb249a0-6e53-46f0-9d42-bd566c77496b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.641011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb249a0-6e53-46f0-9d42-bd566c77496b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.641288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb249a0-6e53-46f0-9d42-bd566c77496b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.642558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb249a0-6e53-46f0-9d42-bd566c77496b None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=fd85838dd97c4d74995302afe060388c) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:30.642558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb249a0-6e53-46f0-9d42-bd566c77496b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:30.643965 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb249a0-6e53-46f0-9d42-bd566c77496b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:30.646134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:30.654343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 386/769] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:30 2022] GET /identity/v3/role_assignments?user.id=fd85838dd97c4d74995302afe060388c&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:30.763096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364090c-1c6b-4301-bc41-c964a46752ef None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:30.815803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48b89e-b9cd-41ac-9fc9-d9ed51720015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:30.816504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48b89e-b9cd-41ac-9fc9-d9ed51720015 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.816784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48b89e-b9cd-41ac-9fc9-d9ed51720015 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.817061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48b89e-b9cd-41ac-9fc9-d9ed51720015 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.817870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48b89e-b9cd-41ac-9fc9-d9ed51720015 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:30.818337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48b89e-b9cd-41ac-9fc9-d9ed51720015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:30.819820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48b89e-b9cd-41ac-9fc9-d9ed51720015 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:30.821206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:30.826387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 384/770] 10.222.0.42 () {56 vars in 1124 bytes} [Fri Jun 3 09:38:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:30.832562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb249a0-6e53-46f0-9d42-bd566c77496b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:30.943099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb249a0-6e53-46f0-9d42-bd566c77496b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:30.978727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e034e0a-35d1-42e4-a8c9-e49ea2aec190 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.979248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e034e0a-35d1-42e4-a8c9-e49ea2aec190 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.979248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e034e0a-35d1-42e4-a8c9-e49ea2aec190 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.980611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 385/771] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:38:30.982373 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649abae5-9558-4084-95ca-17dfafe632fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:30.983076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649abae5-9558-4084-95ca-17dfafe632fe None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.983362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649abae5-9558-4084-95ca-17dfafe632fe None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.983635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649abae5-9558-4084-95ca-17dfafe632fe None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.984357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649abae5-9558-4084-95ca-17dfafe632fe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:30.984664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649abae5-9558-4084-95ca-17dfafe632fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:30.986005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b9ef5-afec-49ff-b97a-03baf531666c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:30.986117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649abae5-9558-4084-95ca-17dfafe632fe None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:30.986617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b9ef5-afec-49ff-b97a-03baf531666c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:30.987093 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b9ef5-afec-49ff-b97a-03baf531666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:30.993214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 387/772] 10.222.0.42 () {56 vars in 1124 bytes} [Fri Jun 3 09:38:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:30.998975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649abae5-9558-4084-95ca-17dfafe632fe None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:31.005992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-172b9ef5-afec-49ff-b97a-03baf531666c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:31.007447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-172b9ef5-afec-49ff-b97a-03baf531666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:31.098560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 386/773] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:31.121369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b9ef5-afec-49ff-b97a-03baf531666c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:31.126233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:31.220108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649abae5-9558-4084-95ca-17dfafe632fe None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:31.225251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b9ef5-afec-49ff-b97a-03baf531666c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:31.262011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:31.276350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3a441-a124-419a-8fe2-142bc8cecf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:31.277162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3a441-a124-419a-8fe2-142bc8cecf7f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:31.277435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3a441-a124-419a-8fe2-142bc8cecf7f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:31.277708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3a441-a124-419a-8fe2-142bc8cecf7f None admin] PATH_INFO: `/v3/services/network` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:31.278518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3a441-a124-419a-8fe2-142bc8cecf7f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:31.279893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3a441-a124-419a-8fe2-142bc8cecf7f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:31.286743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fe548-240c-465e-902b-5eef65fa37f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:31.287526 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fe548-240c-465e-902b-5eef65fa37f0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:31.287811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fe548-240c-465e-902b-5eef65fa37f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:31.288083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fe548-240c-465e-902b-5eef65fa37f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:31.288875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fe548-240c-465e-902b-5eef65fa37f0 None admin] RBAC: Adding query filter params (interface=public, service_id=ee7c2b7e15dd4425934e057a0624cd5d, region_id=RegionOne) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:31.289185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fe548-240c-465e-902b-5eef65fa37f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:31.290817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fe548-240c-465e-902b-5eef65fa37f0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-a5f3a441-a124-419a-8fe2-142bc8cecf7f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:31.304017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:31.305006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 09:38:31.305852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:31.310889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 387/774] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:31 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:31.313743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 388/775] 10.222.0.42 () {56 vars in 1251 bytes} [Fri Jun 3 09:38:30 2022] GET /identity/v3/endpoints?service_id=ee7c2b7e15dd4425934e057a0624cd5d&interface=public®ion_id=RegionOne => generated 194 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:31.314667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3a441-a124-419a-8fe2-142bc8cecf7f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:31.319906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fe548-240c-465e-902b-5eef65fa37f0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:31.487731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fe548-240c-465e-902b-5eef65fa37f0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:31.492334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3a441-a124-419a-8fe2-142bc8cecf7f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:31.536299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:31.549628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c250ea-4dba-4513-937d-65cbb5d7ac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:31.550325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c250ea-4dba-4513-937d-65cbb5d7ac49 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:31.550607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c250ea-4dba-4513-937d-65cbb5d7ac49 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:31.550886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c250ea-4dba-4513-937d-65cbb5d7ac49 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:31.551586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c250ea-4dba-4513-937d-65cbb5d7ac49 None admin] RBAC: Adding query filter params (name=network) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:31.551890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c250ea-4dba-4513-937d-65cbb5d7ac49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:31.553833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c250ea-4dba-4513-937d-65cbb5d7ac49 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:31.555013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b056e4-313e-479a-bd51-78dc3bb0c127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:31.559884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b056e4-313e-479a-bd51-78dc3bb0c127 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:31.559884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b056e4-313e-479a-bd51-78dc3bb0c127 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:31.559884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b056e4-313e-479a-bd51-78dc3bb0c127 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:31.559884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b056e4-313e-479a-bd51-78dc3bb0c127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:31.559884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b056e4-313e-479a-bd51-78dc3bb0c127 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:31.565797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 389/776] 10.222.0.42 () {56 vars in 1087 bytes} [Fri Jun 3 09:38:31 2022] GET /identity/v3/services => generated 772 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:31.568270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 388/777] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:31.574064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b056e4-313e-479a-bd51-78dc3bb0c127 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:31.717251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b056e4-313e-479a-bd51-78dc3bb0c127 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:31.751536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca5f55-141b-416a-9fa5-065d40b5a591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:31.752122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca5f55-141b-416a-9fa5-065d40b5a591 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:31.752403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca5f55-141b-416a-9fa5-065d40b5a591 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:31.752679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca5f55-141b-416a-9fa5-065d40b5a591 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:31.753386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca5f55-141b-416a-9fa5-065d40b5a591 None admin] RBAC: Adding query filter params (type=network) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:31.753692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca5f55-141b-416a-9fa5-065d40b5a591 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:31.755092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca5f55-141b-416a-9fa5-065d40b5a591 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:31.760853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 390/778] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:31.766657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c250ea-4dba-4513-937d-65cbb5d7ac49 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:31.844537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:31.902616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c250ea-4dba-4513-937d-65cbb5d7ac49 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:31.947230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9224f9b-f852-424b-9d5e-4a285d84f362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:31.947897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9224f9b-f852-424b-9d5e-4a285d84f362 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:31.948178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9224f9b-f852-424b-9d5e-4a285d84f362 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:31.948449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9224f9b-f852-424b-9d5e-4a285d84f362 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:31.949232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9224f9b-f852-424b-9d5e-4a285d84f362 None admin] RBAC: Adding query filter params (interface=public, service_id=2731184f7d0e48c29490182fa05a3592, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:31.949540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9224f9b-f852-424b-9d5e-4a285d84f362 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:31.950949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9224f9b-f852-424b-9d5e-4a285d84f362 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:31.960801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 389/779] 10.222.0.42 () {56 vars in 1251 bytes} [Fri Jun 3 09:38:31 2022] GET /identity/v3/endpoints?service_id=2731184f7d0e48c29490182fa05a3592&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:31.966060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca5f55-141b-416a-9fa5-065d40b5a591 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:32.052876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:32.104692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca5f55-141b-416a-9fa5-065d40b5a591 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:32.140176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcb109-c0a6-494c-adb5-dc7704d700d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:32.140833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb109-c0a6-494c-adb5-dc7704d700d0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:32.141113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb109-c0a6-494c-adb5-dc7704d700d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:32.141390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb109-c0a6-494c-adb5-dc7704d700d0 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:32.142097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb109-c0a6-494c-adb5-dc7704d700d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:32.143467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb109-c0a6-494c-adb5-dc7704d700d0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:32.149603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 391/780] 10.222.0.42 () {56 vars in 1087 bytes} [Fri Jun 3 09:38:31 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:32.173198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18738876-5bcc-4a08-9043-4a34c751344c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:32.173749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18738876-5bcc-4a08-9043-4a34c751344c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:32.173749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18738876-5bcc-4a08-9043-4a34c751344c None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:32.175206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 390/781] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:32.181315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ec7bf-bad6-4a37-a6da-dd5c85b33e7a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:32.181637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ec7bf-bad6-4a37-a6da-dd5c85b33e7a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:32.181907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ec7bf-bad6-4a37-a6da-dd5c85b33e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:32.200125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5a1ec7bf-bad6-4a37-a6da-dd5c85b33e7a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:32.201327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5a1ec7bf-bad6-4a37-a6da-dd5c85b33e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:32.279018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 392/782] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:32.306512 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18738876-5bcc-4a08-9043-4a34c751344c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:32.449098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18738876-5bcc-4a08-9043-4a34c751344c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:32.469870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:32.484464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df1d4e-b4e6-427f-a43e-27950d3e37a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:32.485262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df1d4e-b4e6-427f-a43e-27950d3e37a2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:32.485548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df1d4e-b4e6-427f-a43e-27950d3e37a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:32.485821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df1d4e-b4e6-427f-a43e-27950d3e37a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:32.497793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df1d4e-b4e6-427f-a43e-27950d3e37a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d3df1d4e-b4e6-427f-a43e-27950d3e37a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:32.505875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:32.506857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:32.507676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 391/783] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:32.512896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ec7bf-bad6-4a37-a6da-dd5c85b33e7a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:32.569079 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:32.633110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ec7bf-bad6-4a37-a6da-dd5c85b33e7a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:32.683504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351c988-a67c-4668-9419-a514f4f56373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:32.684176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351c988-a67c-4668-9419-a514f4f56373 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:32.684454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351c988-a67c-4668-9419-a514f4f56373 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:32.684727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351c988-a67c-4668-9419-a514f4f56373 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:32.685472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351c988-a67c-4668-9419-a514f4f56373 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:32.685774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351c988-a67c-4668-9419-a514f4f56373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:32.687176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351c988-a67c-4668-9419-a514f4f56373 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:32.701681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 393/784] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:32.707114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df1d4e-b4e6-427f-a43e-27950d3e37a2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:32.838401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df1d4e-b4e6-427f-a43e-27950d3e37a2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:32.877331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afb320-8d74-48ee-91d4-68aad477c07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:32.878160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afb320-8d74-48ee-91d4-68aad477c07f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:32.878439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afb320-8d74-48ee-91d4-68aad477c07f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:32.878713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afb320-8d74-48ee-91d4-68aad477c07f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:32.889877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afb320-8d74-48ee-91d4-68aad477c07f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:32.891389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afb320-8d74-48ee-91d4-68aad477c07f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-e5afb320-8d74-48ee-91d4-68aad477c07f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:32.915678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:32.916626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:32.917498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:32.917498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:32.917498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:32.917498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:32.917498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 392/785] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:32.922717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351c988-a67c-4668-9419-a514f4f56373 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:33.049305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351c988-a67c-4668-9419-a514f4f56373 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:33.061600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:33.085946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7395a37-1d1f-4935-9585-b819cf26d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:33.086612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7395a37-1d1f-4935-9585-b819cf26d185 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:33.086940 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7395a37-1d1f-4935-9585-b819cf26d185 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:33.087217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7395a37-1d1f-4935-9585-b819cf26d185 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:33.087938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7395a37-1d1f-4935-9585-b819cf26d185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:33.088305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7395a37-1d1f-4935-9585-b819cf26d185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:33.089779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7395a37-1d1f-4935-9585-b819cf26d185 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:33.102740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 394/786] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:33.107994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afb320-8d74-48ee-91d4-68aad477c07f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:33.139535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:33.209541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afb320-8d74-48ee-91d4-68aad477c07f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:33.243778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c64480-bcd4-4b57-806d-e8a7bdd56908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:33.244625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64480-bcd4-4b57-806d-e8a7bdd56908 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:33.244906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64480-bcd4-4b57-806d-e8a7bdd56908 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:33.245221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c64480-bcd4-4b57-806d-e8a7bdd56908 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:33.252888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c64480-bcd4-4b57-806d-e8a7bdd56908 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:33.254368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c64480-bcd4-4b57-806d-e8a7bdd56908 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d5c64480-bcd4-4b57-806d-e8a7bdd56908 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:33.268445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:33.269424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:38:33.270391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:33.273002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 393/787] 10.222.0.42 () {56 vars in 1093 bytes} [Fri Jun 3 09:38:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:33.276085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7395a37-1d1f-4935-9585-b819cf26d185 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:33.393106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7395a37-1d1f-4935-9585-b819cf26d185 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:33.434554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940b53a-9222-4617-b465-5f99aa364a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:33.435227 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940b53a-9222-4617-b465-5f99aa364a34 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:33.435508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940b53a-9222-4617-b465-5f99aa364a34 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:33.435782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940b53a-9222-4617-b465-5f99aa364a34 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:33.436621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940b53a-9222-4617-b465-5f99aa364a34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:33.436925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940b53a-9222-4617-b465-5f99aa364a34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:33.442453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940b53a-9222-4617-b465-5f99aa364a34 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:33.468101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 395/788] 10.222.0.42 () {56 vars in 1133 bytes} [Fri Jun 3 09:38:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:33.470030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ed48-d9b5-45aa-8740-4f58a70d0da0 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:33.470372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ed48-d9b5-45aa-8740-4f58a70d0da0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:33.477656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ed48-d9b5-45aa-8740-4f58a70d0da0 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:33.477656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 394/789] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:33.477841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940b53a-9222-4617-b465-5f99aa364a34 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:33.506815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:33.508666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166b0a8-7f38-4e0d-996b-f14721bd9516 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:33.508954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166b0a8-7f38-4e0d-996b-f14721bd9516 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:33.509232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166b0a8-7f38-4e0d-996b-f14721bd9516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:33.538604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1166b0a8-7f38-4e0d-996b-f14721bd9516 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:33.540039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1166b0a8-7f38-4e0d-996b-f14721bd9516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:33.582528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940b53a-9222-4617-b465-5f99aa364a34 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:33.617860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3695d6b-897f-4854-ac3c-c5f4c4edb11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:33.618830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3695d6b-897f-4854-ac3c-c5f4c4edb11d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:33.619447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3695d6b-897f-4854-ac3c-c5f4c4edb11d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:33.619908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3695d6b-897f-4854-ac3c-c5f4c4edb11d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:33.639952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 395/790] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:33.639952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f767a-74a7-4333-8592-59183e3e2c79 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:33.639952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f767a-74a7-4333-8592-59183e3e2c79 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:33.639952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f767a-74a7-4333-8592-59183e3e2c79 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:33.644151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 396/791] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:33.645507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39f275-5291-47d2-ac38-bdc0fe03595a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:33.645908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39f275-5291-47d2-ac38-bdc0fe03595a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:33.646350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39f275-5291-47d2-ac38-bdc0fe03595a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:33.666374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fb39f275-5291-47d2-ac38-bdc0fe03595a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:33.667984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fb39f275-5291-47d2-ac38-bdc0fe03595a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:33.688739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:33.735309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3695d6b-897f-4854-ac3c-c5f4c4edb11d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:33.735309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3695d6b-897f-4854-ac3c-c5f4c4edb11d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:33.753977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 397/792] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:33.756811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39f275-5291-47d2-ac38-bdc0fe03595a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:33.782369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:33.869775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39f275-5291-47d2-ac38-bdc0fe03595a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:33.879109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 396/793] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:38:33.880586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3695d6b-897f-4854-ac3c-c5f4c4edb11d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:33.916398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a2447-5c2f-43e6-a461-ddf44060988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:33.917147 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a2447-5c2f-43e6-a461-ddf44060988e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:33.917429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a2447-5c2f-43e6-a461-ddf44060988e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:33.917707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a2447-5c2f-43e6-a461-ddf44060988e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:33.932844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a2447-5c2f-43e6-a461-ddf44060988e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c50a2447-5c2f-43e6-a461-ddf44060988e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:33.943701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:33.944657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:33.945503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:33.950003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 398/794] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:33.951089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a2447-5c2f-43e6-a461-ddf44060988e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:33.967270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:34.015364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3695d6b-897f-4854-ac3c-c5f4c4edb11d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.057646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a2447-5c2f-43e6-a461-ddf44060988e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.059348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:34.072150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc2103-6b7d-429d-9491-ed46f509ead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.074438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc2103-6b7d-429d-9491-ed46f509ead9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.075008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc2103-6b7d-429d-9491-ed46f509ead9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.075688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc2103-6b7d-429d-9491-ed46f509ead9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.091594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc2103-6b7d-429d-9491-ed46f509ead9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:34.094310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de0dbf-3ea4-4951-82ff-1f5c41c54543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.095209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de0dbf-3ea4-4951-82ff-1f5c41c54543 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.095594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de0dbf-3ea4-4951-82ff-1f5c41c54543 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.095969 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de0dbf-3ea4-4951-82ff-1f5c41c54543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6bdc2103-6b7d-429d-9491-ed46f509ead9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:34.102178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:34.103293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:34.104318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:34.106453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 397/795] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:34.107333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc2103-6b7d-429d-9491-ed46f509ead9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.114520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de0dbf-3ea4-4951-82ff-1f5c41c54543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:34.117189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de0dbf-3ea4-4951-82ff-1f5c41c54543 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c9de0dbf-3ea4-4951-82ff-1f5c41c54543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:34.139909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:34.141008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:34.142108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:34.142108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:34.142108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:34.142108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:34.144233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 399/796] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:34.145415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de0dbf-3ea4-4951-82ff-1f5c41c54543 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.220498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc2103-6b7d-429d-9491-ed46f509ead9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.227913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:34.255899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca288c-1a48-457e-b6b9-f1f3918f7ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.256568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca288c-1a48-457e-b6b9-f1f3918f7ad4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.256847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca288c-1a48-457e-b6b9-f1f3918f7ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.257120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca288c-1a48-457e-b6b9-f1f3918f7ad4 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.257861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca288c-1a48-457e-b6b9-f1f3918f7ad4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:34.258517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca288c-1a48-457e-b6b9-f1f3918f7ad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:34.259662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca288c-1a48-457e-b6b9-f1f3918f7ad4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:34.268249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 398/797] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:34.269125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca288c-1a48-457e-b6b9-f1f3918f7ad4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.272120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de0dbf-3ea4-4951-82ff-1f5c41c54543 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.308996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed377-0a02-4793-bf49-3c7e7260d1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.310116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed377-0a02-4793-bf49-3c7e7260d1af None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.310638 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed377-0a02-4793-bf49-3c7e7260d1af None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.311231 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed377-0a02-4793-bf49-3c7e7260d1af None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.312196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ed377-0a02-4793-bf49-3c7e7260d1af None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:34.312795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ed377-0a02-4793-bf49-3c7e7260d1af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:34.314419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ed377-0a02-4793-bf49-3c7e7260d1af None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:34.319982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:34.330419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 400/798] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:34.331564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed377-0a02-4793-bf49-3c7e7260d1af None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.372473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca288c-1a48-457e-b6b9-f1f3918f7ad4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.413289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d328f-d41e-4987-b099-82b000ab51fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.414415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d328f-d41e-4987-b099-82b000ab51fc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.414599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d328f-d41e-4987-b099-82b000ab51fc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.414861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d328f-d41e-4987-b099-82b000ab51fc None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.415579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d328f-d41e-4987-b099-82b000ab51fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:34.415888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d328f-d41e-4987-b099-82b000ab51fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:34.417254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d328f-d41e-4987-b099-82b000ab51fc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:34.431862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 399/799] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:34.432787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d328f-d41e-4987-b099-82b000ab51fc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.452211 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed377-0a02-4793-bf49-3c7e7260d1af None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.479814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:34.489035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f15ab-6634-411d-b3d9-36741f72c75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.490693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f15ab-6634-411d-b3d9-36741f72c75f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.491179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f15ab-6634-411d-b3d9-36741f72c75f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.491859 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f15ab-6634-411d-b3d9-36741f72c75f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.503526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f15ab-6634-411d-b3d9-36741f72c75f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:34.505456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f15ab-6634-411d-b3d9-36741f72c75f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-591f15ab-6634-411d-b3d9-36741f72c75f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:34.523845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:34.528594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:34.529561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:34.529561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:34.529561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:34.529561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:34.529561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 401/800] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:34.529561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f15ab-6634-411d-b3d9-36741f72c75f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.547664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d328f-d41e-4987-b099-82b000ab51fc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.559292 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:34.585255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377826a-0b4c-4d7a-a1fa-4d55f571c8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.586344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377826a-0b4c-4d7a-a1fa-4d55f571c8f4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.586629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377826a-0b4c-4d7a-a1fa-4d55f571c8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.586904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377826a-0b4c-4d7a-a1fa-4d55f571c8f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.604203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377826a-0b4c-4d7a-a1fa-4d55f571c8f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:34.605711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377826a-0b4c-4d7a-a1fa-4d55f571c8f4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-8377826a-0b4c-4d7a-a1fa-4d55f571c8f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:34.626727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:34.627893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:34.628874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:34.628874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:34.628874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:34.628874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:34.628874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 400/801] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:34.629580 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddcd79-d652-46f7-9c2e-b1a2333cc830 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.629862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddcd79-d652-46f7-9c2e-b1a2333cc830 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.630156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddcd79-d652-46f7-9c2e-b1a2333cc830 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.635389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 401/802] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:34.636153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddcd79-d652-46f7-9c2e-b1a2333cc830 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.643028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f15ab-6634-411d-b3d9-36741f72c75f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.678643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30808224-4c73-45bb-a47c-a140da67ad09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.679437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30808224-4c73-45bb-a47c-a140da67ad09 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.679721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30808224-4c73-45bb-a47c-a140da67ad09 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.679999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30808224-4c73-45bb-a47c-a140da67ad09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.686793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30808224-4c73-45bb-a47c-a140da67ad09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:34.688239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30808224-4c73-45bb-a47c-a140da67ad09 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-30808224-4c73-45bb-a47c-a140da67ad09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:34.702368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:34.703248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:34.704136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:34.704136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:34.704136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:34.704136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:34.704136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:34.704136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:34.704136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:34.704136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:34.704136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 402/803] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:34.704606 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30808224-4c73-45bb-a47c-a140da67ad09 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.792431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddcd79-d652-46f7-9c2e-b1a2333cc830 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.792744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:34.799379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c74ba-1356-43e9-9b0a-6c6e1d47ccc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.806422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c74ba-1356-43e9-9b0a-6c6e1d47ccc0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.806422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c74ba-1356-43e9-9b0a-6c6e1d47ccc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.806422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c74ba-1356-43e9-9b0a-6c6e1d47ccc0 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.806422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c74ba-1356-43e9-9b0a-6c6e1d47ccc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:34.806422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c74ba-1356-43e9-9b0a-6c6e1d47ccc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:34.806422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c74ba-1356-43e9-9b0a-6c6e1d47ccc0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:34.820625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 402/804] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:34.820625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c542d67-b720-4920-8145-ee160a84bae9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.820625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c542d67-b720-4920-8145-ee160a84bae9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.820625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c542d67-b720-4920-8145-ee160a84bae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.832739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30808224-4c73-45bb-a47c-a140da67ad09 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:34.835613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3c542d67-b720-4920-8145-ee160a84bae9 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:34.836843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3c542d67-b720-4920-8145-ee160a84bae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:34.854007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:34.868730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f5796-a58e-4c54-bd30-a9fc0441edb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:34.869396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f5796-a58e-4c54-bd30-a9fc0441edb1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:34.869696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f5796-a58e-4c54-bd30-a9fc0441edb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:34.869990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f5796-a58e-4c54-bd30-a9fc0441edb1 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:34.870711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f5796-a58e-4c54-bd30-a9fc0441edb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:34.871020 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f5796-a58e-4c54-bd30-a9fc0441edb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:34.872395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f5796-a58e-4c54-bd30-a9fc0441edb1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:34.883864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 403/805] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:34.888889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f5796-a58e-4c54-bd30-a9fc0441edb1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.918157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 403/806] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:34.918891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c542d67-b720-4920-8145-ee160a84bae9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:34.989078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:34.992678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f5796-a58e-4c54-bd30-a9fc0441edb1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.021945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c542d67-b720-4920-8145-ee160a84bae9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.026479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ec95b-50d9-4167-b3e5-8e62964a524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.027141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ec95b-50d9-4167-b3e5-8e62964a524b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.027429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ec95b-50d9-4167-b3e5-8e62964a524b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.027707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ec95b-50d9-4167-b3e5-8e62964a524b None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.028485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ec95b-50d9-4167-b3e5-8e62964a524b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:35.028794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ec95b-50d9-4167-b3e5-8e62964a524b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.030249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ec95b-50d9-4167-b3e5-8e62964a524b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.042418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 404/807] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:35.043205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ec95b-50d9-4167-b3e5-8e62964a524b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.055001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdeb8d8-2b39-4564-abf6-4a1fcaa472ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.055814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdeb8d8-2b39-4564-abf6-4a1fcaa472ba None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.056093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdeb8d8-2b39-4564-abf6-4a1fcaa472ba None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.056366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdeb8d8-2b39-4564-abf6-4a1fcaa472ba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.063505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdeb8d8-2b39-4564-abf6-4a1fcaa472ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.064943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdeb8d8-2b39-4564-abf6-4a1fcaa472ba None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-efdeb8d8-2b39-4564-abf6-4a1fcaa472ba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:35.080163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:35.081872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 404/808] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:38:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:35.083184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdeb8d8-2b39-4564-abf6-4a1fcaa472ba None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.096659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:35.148769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ec95b-50d9-4167-b3e5-8e62964a524b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.178722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdeb8d8-2b39-4564-abf6-4a1fcaa472ba None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.181565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66183617-b042-45bd-97ad-258f1cf5dd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.182379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66183617-b042-45bd-97ad-258f1cf5dd0e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.182663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66183617-b042-45bd-97ad-258f1cf5dd0e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.182937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66183617-b042-45bd-97ad-258f1cf5dd0e None admin] PATH_INFO: `/v3/users/swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.189877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66183617-b042-45bd-97ad-258f1cf5dd0e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.191365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66183617-b042-45bd-97ad-258f1cf5dd0e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-66183617-b042-45bd-97ad-258f1cf5dd0e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:35.206134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:35.207212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:35.208217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 405/809] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:35.209561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf96be1-61b6-4b04-9a7d-c034f2b25e95 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.209844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf96be1-61b6-4b04-9a7d-c034f2b25e95 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.210151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf96be1-61b6-4b04-9a7d-c034f2b25e95 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.211410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 406/810] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:35.212102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf96be1-61b6-4b04-9a7d-c034f2b25e95 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.219025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008dfa5-5bb0-44f1-98d2-93de5245f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.219840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008dfa5-5bb0-44f1-98d2-93de5245f74c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.220124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008dfa5-5bb0-44f1-98d2-93de5245f74c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.220403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008dfa5-5bb0-44f1-98d2-93de5245f74c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.221057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008dfa5-5bb0-44f1-98d2-93de5245f74c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.222484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008dfa5-5bb0-44f1-98d2-93de5245f74c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.230637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-8008dfa5-5bb0-44f1-98d2-93de5245f74c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:35.236575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:35.237733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:35.238822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 405/811] 10.222.0.42 () {56 vars in 1129 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:35.239869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008dfa5-5bb0-44f1-98d2-93de5245f74c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.315352 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:35.317523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf96be1-61b6-4b04-9a7d-c034f2b25e95 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.337273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008dfa5-5bb0-44f1-98d2-93de5245f74c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.352880 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5b0f2-65cc-40ec-b79b-154e6f395d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.353526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5b0f2-65cc-40ec-b79b-154e6f395d57 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.353814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5b0f2-65cc-40ec-b79b-154e6f395d57 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.354118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5b0f2-65cc-40ec-b79b-154e6f395d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.354975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5b0f2-65cc-40ec-b79b-154e6f395d57 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=dfe8ef12db914a9693ef470f1706bb5a) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:35.355285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5b0f2-65cc-40ec-b79b-154e6f395d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.356721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5b0f2-65cc-40ec-b79b-154e6f395d57 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.362384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 407/812] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/role_assignments?user.id=dfe8ef12db914a9693ef470f1706bb5a&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=fe4d636f5c4842f18569347911e6cdec => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:35.363148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5b0f2-65cc-40ec-b79b-154e6f395d57 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.379619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a825ac-559b-4233-8583-85db38133ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.379999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a825ac-559b-4233-8583-85db38133ca0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.380267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a825ac-559b-4233-8583-85db38133ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.380611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a825ac-559b-4233-8583-85db38133ca0 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.381988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a825ac-559b-4233-8583-85db38133ca0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:35.382310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a825ac-559b-4233-8583-85db38133ca0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.383828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a825ac-559b-4233-8583-85db38133ca0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.405925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 406/813] 10.222.0.42 () {56 vars in 1149 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:35.411973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9e126-4aca-43bc-be98-1e02fcbfcdbb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.412364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9e126-4aca-43bc-be98-1e02fcbfcdbb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.412767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9e126-4aca-43bc-be98-1e02fcbfcdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.431850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b6c9e126-4aca-43bc-be98-1e02fcbfcdbb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:35.433200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b6c9e126-4aca-43bc-be98-1e02fcbfcdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:35.456238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:35.479334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5b0f2-65cc-40ec-b79b-154e6f395d57 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.508665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 407/814] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:35.509599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9e126-4aca-43bc-be98-1e02fcbfcdbb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.540917 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15a6c1-4a89-414e-89a3-e2a7fa93b689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.541608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15a6c1-4a89-414e-89a3-e2a7fa93b689 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.541922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15a6c1-4a89-414e-89a3-e2a7fa93b689 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.542206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15a6c1-4a89-414e-89a3-e2a7fa93b689 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.543063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15a6c1-4a89-414e-89a3-e2a7fa93b689 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:35.543377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15a6c1-4a89-414e-89a3-e2a7fa93b689 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.544889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15a6c1-4a89-414e-89a3-e2a7fa93b689 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.563937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 408/815] 10.222.0.42 () {56 vars in 1135 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:35.564655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15a6c1-4a89-414e-89a3-e2a7fa93b689 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.588378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:35.630293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9e126-4aca-43bc-be98-1e02fcbfcdbb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.666910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc58b2a-8087-4b5d-a1e8-01abd5dcfa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.667569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc58b2a-8087-4b5d-a1e8-01abd5dcfa66 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.667849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc58b2a-8087-4b5d-a1e8-01abd5dcfa66 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.668123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc58b2a-8087-4b5d-a1e8-01abd5dcfa66 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.668816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc58b2a-8087-4b5d-a1e8-01abd5dcfa66 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:35.669121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc58b2a-8087-4b5d-a1e8-01abd5dcfa66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.670524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc58b2a-8087-4b5d-a1e8-01abd5dcfa66 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.675130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15a6c1-4a89-414e-89a3-e2a7fa93b689 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.675253 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 408/816] 10.222.0.42 () {56 vars in 1124 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:35.675959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc58b2a-8087-4b5d-a1e8-01abd5dcfa66 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.718159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ab52e-2a3b-498e-900b-e70baf2e9e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.719013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ab52e-2a3b-498e-900b-e70baf2e9e94 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.725906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ab52e-2a3b-498e-900b-e70baf2e9e94 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.725906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ab52e-2a3b-498e-900b-e70baf2e9e94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.729552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:35.732143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ab52e-2a3b-498e-900b-e70baf2e9e94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.733832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ab52e-2a3b-498e-900b-e70baf2e9e94 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-070ab52e-2a3b-498e-900b-e70baf2e9e94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:35.751553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:35.752464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:35.753367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:35.753367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:35.753367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:35.753367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:35.753367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 409/817] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:35.753657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ab52e-2a3b-498e-900b-e70baf2e9e94 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.797259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc58b2a-8087-4b5d-a1e8-01abd5dcfa66 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.824460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:35.832056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc7554-4cac-4d2a-8022-ca8489877366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.836462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc7554-4cac-4d2a-8022-ca8489877366 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.836462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc7554-4cac-4d2a-8022-ca8489877366 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.836462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc7554-4cac-4d2a-8022-ca8489877366 None admin] PATH_INFO: `/v3/services/network` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.836462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc7554-4cac-4d2a-8022-ca8489877366 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.836462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc7554-4cac-4d2a-8022-ca8489877366 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-cbdc7554-4cac-4d2a-8022-ca8489877366 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:35.848157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:35.849340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:35.850339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 409/818] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:35.851113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc7554-4cac-4d2a-8022-ca8489877366 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.867386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ab52e-2a3b-498e-900b-e70baf2e9e94 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.903380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2a827-1184-4331-b0b8-b08d59cecf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:35.904178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2a827-1184-4331-b0b8-b08d59cecf98 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:35.904463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2a827-1184-4331-b0b8-b08d59cecf98 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:35.904741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2a827-1184-4331-b0b8-b08d59cecf98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:35.915891 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2a827-1184-4331-b0b8-b08d59cecf98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:35.917455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2a827-1184-4331-b0b8-b08d59cecf98 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0eb2a827-1184-4331-b0b8-b08d59cecf98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:35.941306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:35.942295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:35.943258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:35.943258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:35.943258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:35.943258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:35.943258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 410/819] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:35.943545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2a827-1184-4331-b0b8-b08d59cecf98 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:35.959711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc7554-4cac-4d2a-8022-ca8489877366 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:35.991106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:36.000910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087972e0-41a1-4094-91b1-4178374b434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.002279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087972e0-41a1-4094-91b1-4178374b434c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.002279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087972e0-41a1-4094-91b1-4178374b434c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.002279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087972e0-41a1-4094-91b1-4178374b434c None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.002882 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087972e0-41a1-4094-91b1-4178374b434c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:36.003144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087972e0-41a1-4094-91b1-4178374b434c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.004546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087972e0-41a1-4094-91b1-4178374b434c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.014830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 410/820] 10.222.0.42 () {56 vars in 1124 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:36.014830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087972e0-41a1-4094-91b1-4178374b434c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.046740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2a827-1184-4331-b0b8-b08d59cecf98 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:36.071907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:36.081635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8e0a0-ebc7-4c4b-b746-22c5e7ca57dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.087628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8e0a0-ebc7-4c4b-b746-22c5e7ca57dd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.087628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8e0a0-ebc7-4c4b-b746-22c5e7ca57dd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.087628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8e0a0-ebc7-4c4b-b746-22c5e7ca57dd None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.087628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8e0a0-ebc7-4c4b-b746-22c5e7ca57dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:36.087628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8e0a0-ebc7-4c4b-b746-22c5e7ca57dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.087628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8e0a0-ebc7-4c4b-b746-22c5e7ca57dd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.098281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 411/821] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:36.099514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8e0a0-ebc7-4c4b-b746-22c5e7ca57dd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.122753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087972e0-41a1-4094-91b1-4178374b434c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:36.160115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b1e06-daa7-449f-bb21-b5da985d4b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.161393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b1e06-daa7-449f-bb21-b5da985d4b0f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.161906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b1e06-daa7-449f-bb21-b5da985d4b0f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.162266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b1e06-daa7-449f-bb21-b5da985d4b0f None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.163205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b1e06-daa7-449f-bb21-b5da985d4b0f None admin] RBAC: Adding query filter params (name=network) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:36.163888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b1e06-daa7-449f-bb21-b5da985d4b0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.165869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b1e06-daa7-449f-bb21-b5da985d4b0f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.172670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 411/822] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:36.173467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b1e06-daa7-449f-bb21-b5da985d4b0f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.203561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8e0a0-ebc7-4c4b-b746-22c5e7ca57dd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:36.206936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:36.237651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b017-dcc3-4285-af9d-0e086dd4c010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.238352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b017-dcc3-4285-af9d-0e086dd4c010 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.238640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b017-dcc3-4285-af9d-0e086dd4c010 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.238916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b017-dcc3-4285-af9d-0e086dd4c010 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.240148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80b017-dcc3-4285-af9d-0e086dd4c010 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:36.240457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80b017-dcc3-4285-af9d-0e086dd4c010 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.241840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80b017-dcc3-4285-af9d-0e086dd4c010 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.254624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 412/823] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:36.255482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b017-dcc3-4285-af9d-0e086dd4c010 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.287556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b1e06-daa7-449f-bb21-b5da985d4b0f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:36.308684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:36.322555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b618b5-12a6-4b44-8128-f865183e8fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.323269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b618b5-12a6-4b44-8128-f865183e8fe8 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.323605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b618b5-12a6-4b44-8128-f865183e8fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.323938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b618b5-12a6-4b44-8128-f865183e8fe8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.328434 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b618b5-12a6-4b44-8128-f865183e8fe8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.328434 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b618b5-12a6-4b44-8128-f865183e8fe8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.359360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b017-dcc3-4285-af9d-0e086dd4c010 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:36.393213 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272197d5-7377-4750-bf29-5c2c934a9162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.394028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272197d5-7377-4750-bf29-5c2c934a9162 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.394314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272197d5-7377-4750-bf29-5c2c934a9162 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.394589 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272197d5-7377-4750-bf29-5c2c934a9162 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.401145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272197d5-7377-4750-bf29-5c2c934a9162 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.402617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272197d5-7377-4750-bf29-5c2c934a9162 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-272197d5-7377-4750-bf29-5c2c934a9162 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:36.416730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:36.417818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 413/824] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:36.418982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272197d5-7377-4750-bf29-5c2c934a9162 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.436092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 412/825] 10.222.0.42 () {60 vars in 1144 bytes} [Fri Jun 3 09:38:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:36.437083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b618b5-12a6-4b44-8128-f865183e8fe8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.449749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:36.556341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272197d5-7377-4750-bf29-5c2c934a9162 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:36.596310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b618b5-12a6-4b44-8128-f865183e8fe8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:36.600540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd7632-bf68-4553-af07-e0be02305ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.601573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd7632-bf68-4553-af07-e0be02305ff4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.602113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd7632-bf68-4553-af07-e0be02305ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.602765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd7632-bf68-4553-af07-e0be02305ff4 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.603748 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd7632-bf68-4553-af07-e0be02305ff4 None admin] RBAC: Adding query filter params (type=network) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:36.604266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd7632-bf68-4553-af07-e0be02305ff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.606202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd7632-bf68-4553-af07-e0be02305ff4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.613155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 414/826] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:36.614433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd7632-bf68-4553-af07-e0be02305ff4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.646288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20156dbe-3fe6-4dd5-b02d-7aba3a2d7d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.647097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20156dbe-3fe6-4dd5-b02d-7aba3a2d7d55 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.647380 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20156dbe-3fe6-4dd5-b02d-7aba3a2d7d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.647658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20156dbe-3fe6-4dd5-b02d-7aba3a2d7d55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.655383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20156dbe-3fe6-4dd5-b02d-7aba3a2d7d55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.656846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20156dbe-3fe6-4dd5-b02d-7aba3a2d7d55 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-20156dbe-3fe6-4dd5-b02d-7aba3a2d7d55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:36.674417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:36.675537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 413/827] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:36.676423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20156dbe-3fe6-4dd5-b02d-7aba3a2d7d55 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.719028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:36.738795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:36.754556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd7632-bf68-4553-af07-e0be02305ff4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:36.791744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09589251-8093-421d-9ecd-ce799e2937c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.792708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09589251-8093-421d-9ecd-ce799e2937c1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.793195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09589251-8093-421d-9ecd-ce799e2937c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.793663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09589251-8093-421d-9ecd-ce799e2937c1 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.795123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09589251-8093-421d-9ecd-ce799e2937c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:36.795939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09589251-8093-421d-9ecd-ce799e2937c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.798086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09589251-8093-421d-9ecd-ce799e2937c1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.807294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20156dbe-3fe6-4dd5-b02d-7aba3a2d7d55 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:36.815184 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 415/828] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:36.817828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09589251-8093-421d-9ecd-ce799e2937c1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.851260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a816c-a56c-40f8-9b68-ea2f9169555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:36.852268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a816c-a56c-40f8-9b68-ea2f9169555f None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:36.852635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a816c-a56c-40f8-9b68-ea2f9169555f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:36.852983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a816c-a56c-40f8-9b68-ea2f9169555f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:36.854149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a816c-a56c-40f8-9b68-ea2f9169555f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:36.856340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a816c-a56c-40f8-9b68-ea2f9169555f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:36.934073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 414/829] 10.222.0.42 () {60 vars in 1144 bytes} [Fri Jun 3 09:38:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:36.935320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a816c-a56c-40f8-9b68-ea2f9169555f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:36.944808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09589251-8093-421d-9ecd-ce799e2937c1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:37.028798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311c69f-1049-4b0f-a82e-7d71c4b77065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:37.029474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311c69f-1049-4b0f-a82e-7d71c4b77065 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:37.029748 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311c69f-1049-4b0f-a82e-7d71c4b77065 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:37.030054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311c69f-1049-4b0f-a82e-7d71c4b77065 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:37.030858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311c69f-1049-4b0f-a82e-7d71c4b77065 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:37.031168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311c69f-1049-4b0f-a82e-7d71c4b77065 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:37.032625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311c69f-1049-4b0f-a82e-7d71c4b77065 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:37.056881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 416/830] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:37.059572 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311c69f-1049-4b0f-a82e-7d71c4b77065 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:37.075004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:37.112414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:37.148527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a816c-a56c-40f8-9b68-ea2f9169555f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:37.185644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311c69f-1049-4b0f-a82e-7d71c4b77065 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:37.223412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424f810-d869-4e9e-a35b-57ba605be8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:37.224822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424f810-d869-4e9e-a35b-57ba605be8ae None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:37.225107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424f810-d869-4e9e-a35b-57ba605be8ae None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:37.225381 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424f810-d869-4e9e-a35b-57ba605be8ae None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/37186dc0010247cd936f160b5eb0298b/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:37.227925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424f810-d869-4e9e-a35b-57ba605be8ae None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=37186dc0010247cd936f160b5eb0298b, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:37.229379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424f810-d869-4e9e-a35b-57ba605be8ae None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:37.241220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf3e5c-428e-4149-9fd3-aad7ed8390ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:37.242209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3e5c-428e-4149-9fd3-aad7ed8390ea None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:37.246664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3e5c-428e-4149-9fd3-aad7ed8390ea None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:37.246664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf3e5c-428e-4149-9fd3-aad7ed8390ea None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/fd85838dd97c4d74995302afe060388c/roles/4766b6a3fe4746798f255d0813534a04` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:37.259378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf3e5c-428e-4149-9fd3-aad7ed8390ea None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=fd85838dd97c4d74995302afe060388c, role_id=4766b6a3fe4746798f255d0813534a04)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:37.260827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf3e5c-428e-4149-9fd3-aad7ed8390ea None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:37.300727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 417/831] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:38:37 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/37186dc0010247cd936f160b5eb0298b/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:38:37.330121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 415/832] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:38:36 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/fd85838dd97c4d74995302afe060388c/roles/4766b6a3fe4746798f255d0813534a04 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:38:37.706616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c125d3f-e313-4347-a8d6-6fdae47af53a None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:37.707052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c125d3f-e313-4347-a8d6-6fdae47af53a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:37.707231 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c125d3f-e313-4347-a8d6-6fdae47af53a None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:37.708810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 418/833] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:37.720662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a235c6-7b33-406f-b0b3-c8a718a2c220 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:37.720951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a235c6-7b33-406f-b0b3-c8a718a2c220 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:37.721228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a235c6-7b33-406f-b0b3-c8a718a2c220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:37.738886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e2a235c6-7b33-406f-b0b3-c8a718a2c220 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:37.740621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e2a235c6-7b33-406f-b0b3-c8a718a2c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:37.813765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 416/834] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:37.837425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c125d3f-e313-4347-a8d6-6fdae47af53a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:37.905503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:37.948037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c125d3f-e313-4347-a8d6-6fdae47af53a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:37.982539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52be22-a3c2-4a52-aa7a-f99cccbb1ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:37.983287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52be22-a3c2-4a52-aa7a-f99cccbb1ad7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:37.983575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52be22-a3c2-4a52-aa7a-f99cccbb1ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:37.983858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52be22-a3c2-4a52-aa7a-f99cccbb1ad7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:37.984514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52be22-a3c2-4a52-aa7a-f99cccbb1ad7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:37.985927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee52be22-a3c2-4a52-aa7a-f99cccbb1ad7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ee52be22-a3c2-4a52-aa7a-f99cccbb1ad7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:37.998419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:37.999312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:38.000120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 419/835] 10.222.0.42 () {56 vars in 1132 bytes} [Fri Jun 3 09:38:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:38.006032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a235c6-7b33-406f-b0b3-c8a718a2c220 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:38.073757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:38.111445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a235c6-7b33-406f-b0b3-c8a718a2c220 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:38.148007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac5325-79ed-416b-9f24-03dd432a64b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:38.148678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac5325-79ed-416b-9f24-03dd432a64b0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:38.149291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac5325-79ed-416b-9f24-03dd432a64b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:38.149291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac5325-79ed-416b-9f24-03dd432a64b0 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:38.149956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac5325-79ed-416b-9f24-03dd432a64b0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:38.150267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac5325-79ed-416b-9f24-03dd432a64b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:38.151643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac5325-79ed-416b-9f24-03dd432a64b0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:38.156822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 417/836] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:38.162725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52be22-a3c2-4a52-aa7a-f99cccbb1ad7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:38.276781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52be22-a3c2-4a52-aa7a-f99cccbb1ad7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:38.306268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:38.311619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620d71a-e8df-42cc-99d7-e9975138db13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:38.312289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620d71a-e8df-42cc-99d7-e9975138db13 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:38.312575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620d71a-e8df-42cc-99d7-e9975138db13 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:38.312857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620d71a-e8df-42cc-99d7-e9975138db13 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:38.313602 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620d71a-e8df-42cc-99d7-e9975138db13 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:38.313936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620d71a-e8df-42cc-99d7-e9975138db13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:38.319533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620d71a-e8df-42cc-99d7-e9975138db13 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:38.320441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 420/837] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:38.942358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28db1f3-a9be-4efe-98ac-87c5ad27e706 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:38.942889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28db1f3-a9be-4efe-98ac-87c5ad27e706 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:38.943422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28db1f3-a9be-4efe-98ac-87c5ad27e706 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:38.945323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 418/838] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:38.950749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf78cd-0ea2-4e5d-8707-1922391543d7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:38.951156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf78cd-0ea2-4e5d-8707-1922391543d7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:38.951541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf78cd-0ea2-4e5d-8707-1922391543d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:38.968568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cccf78cd-0ea2-4e5d-8707-1922391543d7 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:38.969694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cccf78cd-0ea2-4e5d-8707-1922391543d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:39.046055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 421/839] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:39.062858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28db1f3-a9be-4efe-98ac-87c5ad27e706 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:39.170175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28db1f3-a9be-4efe-98ac-87c5ad27e706 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:39.201759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:39.207311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd48f27-bd77-4e64-8f71-cf42c9cffb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:39.208118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd48f27-bd77-4e64-8f71-cf42c9cffb20 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:39.208455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd48f27-bd77-4e64-8f71-cf42c9cffb20 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:39.208737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd48f27-bd77-4e64-8f71-cf42c9cffb20 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:39.209398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd48f27-bd77-4e64-8f71-cf42c9cffb20 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:39.210876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd48f27-bd77-4e64-8f71-cf42c9cffb20 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-1cd48f27-bd77-4e64-8f71-cf42c9cffb20 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:39.223154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:39.224177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 09:38:39.224986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:39.229006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 419/840] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:38:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:39.230753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf78cd-0ea2-4e5d-8707-1922391543d7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:39.336724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf78cd-0ea2-4e5d-8707-1922391543d7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:39.362961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:39.376354 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0f32d-84d5-41ec-b501-b02ab461a307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:39.383011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f32d-84d5-41ec-b501-b02ab461a307 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:39.383011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f32d-84d5-41ec-b501-b02ab461a307 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:39.383011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f32d-84d5-41ec-b501-b02ab461a307 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:39.383011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0f32d-84d5-41ec-b501-b02ab461a307 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:39.383011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0f32d-84d5-41ec-b501-b02ab461a307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:39.383011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0f32d-84d5-41ec-b501-b02ab461a307 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:39.390013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 422/841] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:38:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:39.391683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd48f27-bd77-4e64-8f71-cf42c9cffb20 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:39.497138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd48f27-bd77-4e64-8f71-cf42c9cffb20 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:39.531179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2cb03-2b13-4815-9f3c-615421a7c4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:39.531850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2cb03-2b13-4815-9f3c-615421a7c4aa None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:39.532137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2cb03-2b13-4815-9f3c-615421a7c4aa None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:39.532417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2cb03-2b13-4815-9f3c-615421a7c4aa None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:39.533193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2cb03-2b13-4815-9f3c-615421a7c4aa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:39.533506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2cb03-2b13-4815-9f3c-615421a7c4aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:39.535020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2cb03-2b13-4815-9f3c-615421a7c4aa None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:39.544401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 420/842] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:38:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:39.660212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d79d-0fa7-4d64-a9b9-4f255d770040 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:39.660608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d79d-0fa7-4d64-a9b9-4f255d770040 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:39.660789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d79d-0fa7-4d64-a9b9-4f255d770040 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:39.663039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 423/843] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:39.668849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e4f49-63c7-424c-a09a-ff9af76ec0f0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:39.669409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e4f49-63c7-424c-a09a-ff9af76ec0f0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:39.669917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e4f49-63c7-424c-a09a-ff9af76ec0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:39.688563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-927e4f49-63c7-424c-a09a-ff9af76ec0f0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:39.690184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-927e4f49-63c7-424c-a09a-ff9af76ec0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:39.750504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519824f5-10ba-4bf0-b05b-ec9aefa99645 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:39.750693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519824f5-10ba-4bf0-b05b-ec9aefa99645 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:39.750980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519824f5-10ba-4bf0-b05b-ec9aefa99645 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:39.752469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 424/844] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:39.758781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9728bca-a0a6-4200-ba99-b891eac7d13a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:39.759074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9728bca-a0a6-4200-ba99-b891eac7d13a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:39.759355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9728bca-a0a6-4200-ba99-b891eac7d13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:39.760952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:39.772417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 421/845] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:39.780304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a9728bca-a0a6-4200-ba99-b891eac7d13a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:39.783968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a9728bca-a0a6-4200-ba99-b891eac7d13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:39.795608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e4f49-63c7-424c-a09a-ff9af76ec0f0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:39.872126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 425/846] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:39.901582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9728bca-a0a6-4200-ba99-b891eac7d13a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:39.906176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e4f49-63c7-424c-a09a-ff9af76ec0f0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:39.954555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161898ed-7565-43aa-9fd5-9168977d29f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:39.956627 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161898ed-7565-43aa-9fd5-9168977d29f9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:39.957143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161898ed-7565-43aa-9fd5-9168977d29f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:39.957622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161898ed-7565-43aa-9fd5-9168977d29f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:39.972175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161898ed-7565-43aa-9fd5-9168977d29f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-161898ed-7565-43aa-9fd5-9168977d29f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:39.982393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:39.983333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:39.984164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 422/847] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:39.992031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161898ed-7565-43aa-9fd5-9168977d29f9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.016167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9728bca-a0a6-4200-ba99-b891eac7d13a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.038803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:40.049416 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:40.059306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb1366-1332-4e81-a509-38bc9a414242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.060412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb1366-1332-4e81-a509-38bc9a414242 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.060871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb1366-1332-4e81-a509-38bc9a414242 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.061461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb1366-1332-4e81-a509-38bc9a414242 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.073464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb1366-1332-4e81-a509-38bc9a414242 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-36fb1366-1332-4e81-a509-38bc9a414242 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:40.081953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:40.082894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:38:40.084255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:40.087019 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 426/848] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:38:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:40.092433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb1366-1332-4e81-a509-38bc9a414242 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.112307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161898ed-7565-43aa-9fd5-9168977d29f9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.148049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fcf3d-ef38-4272-9e62-1ab97e7a3470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.149102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcf3d-ef38-4272-9e62-1ab97e7a3470 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.149582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcf3d-ef38-4272-9e62-1ab97e7a3470 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.150097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fcf3d-ef38-4272-9e62-1ab97e7a3470 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.151101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fcf3d-ef38-4272-9e62-1ab97e7a3470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:40.151602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fcf3d-ef38-4272-9e62-1ab97e7a3470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:40.153206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fcf3d-ef38-4272-9e62-1ab97e7a3470 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:40.167199 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 423/849] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:40.168381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fcf3d-ef38-4272-9e62-1ab97e7a3470 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.216137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb1366-1332-4e81-a509-38bc9a414242 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.254560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f866f9-0cd6-4002-a286-10e5a84210a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.256129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f866f9-0cd6-4002-a286-10e5a84210a9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.256633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f866f9-0cd6-4002-a286-10e5a84210a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.257116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f866f9-0cd6-4002-a286-10e5a84210a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.258082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f866f9-0cd6-4002-a286-10e5a84210a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:40.258584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f866f9-0cd6-4002-a286-10e5a84210a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:40.260179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f866f9-0cd6-4002-a286-10e5a84210a9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:40.269263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 427/850] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:38:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:40.274898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f866f9-0cd6-4002-a286-10e5a84210a9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.302060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fcf3d-ef38-4272-9e62-1ab97e7a3470 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.321862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:40.338363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:40.353793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472d163-b31b-4cc6-a946-008ac5e2cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.354687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472d163-b31b-4cc6-a946-008ac5e2cd5d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.354974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472d163-b31b-4cc6-a946-008ac5e2cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.355294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472d163-b31b-4cc6-a946-008ac5e2cd5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.371456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472d163-b31b-4cc6-a946-008ac5e2cd5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:40.373258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472d163-b31b-4cc6-a946-008ac5e2cd5d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:40.390105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f866f9-0cd6-4002-a286-10e5a84210a9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6472d163-b31b-4cc6-a946-008ac5e2cd5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:40.396593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:40.397482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:40.398376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:40.398376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:40.398376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:40.398376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:40.402028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 424/851] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:40.403335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472d163-b31b-4cc6-a946-008ac5e2cd5d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.433296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf4d4b-b9c9-4683-9088-933cc96c81da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.434245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4d4b-b9c9-4683-9088-933cc96c81da None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.434430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4d4b-b9c9-4683-9088-933cc96c81da None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.434681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4d4b-b9c9-4683-9088-933cc96c81da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.445955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf4d4b-b9c9-4683-9088-933cc96c81da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:40.447495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf4d4b-b9c9-4683-9088-933cc96c81da None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c6bf4d4b-b9c9-4683-9088-933cc96c81da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:40.465578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:40.474391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:40.475302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:40.475302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:40.475302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:40.475302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:40.475302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 428/852] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:40.475302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf4d4b-b9c9-4683-9088-933cc96c81da None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.521259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472d163-b31b-4cc6-a946-008ac5e2cd5d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.558554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93f840-a5bd-4124-b860-500ffb5bb9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.559639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93f840-a5bd-4124-b860-500ffb5bb9b7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.560194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93f840-a5bd-4124-b860-500ffb5bb9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.560666 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93f840-a5bd-4124-b860-500ffb5bb9b7 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.561625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93f840-a5bd-4124-b860-500ffb5bb9b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:40.562161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93f840-a5bd-4124-b860-500ffb5bb9b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:40.563878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93f840-a5bd-4124-b860-500ffb5bb9b7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:40.568856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:40.581758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf4d4b-b9c9-4683-9088-933cc96c81da None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.582357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 425/853] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:40.583924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93f840-a5bd-4124-b860-500ffb5bb9b7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.609918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:40.614401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07086705-7322-46f9-969e-c456de3d75b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.615613 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07086705-7322-46f9-969e-c456de3d75b5 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.615962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07086705-7322-46f9-969e-c456de3d75b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.616216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07086705-7322-46f9-969e-c456de3d75b5 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.616931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07086705-7322-46f9-969e-c456de3d75b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:40.617233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07086705-7322-46f9-969e-c456de3d75b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:40.618719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07086705-7322-46f9-969e-c456de3d75b5 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:40.635716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 429/854] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:40.636795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07086705-7322-46f9-969e-c456de3d75b5 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.697132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93f840-a5bd-4124-b860-500ffb5bb9b7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.742505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c597c42-7dd9-4b94-ace0-197a14ed459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.743566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c597c42-7dd9-4b94-ace0-197a14ed459f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.744139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c597c42-7dd9-4b94-ace0-197a14ed459f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.744579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c597c42-7dd9-4b94-ace0-197a14ed459f None admin] PATH_INFO: `/v3/users/swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.746567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07086705-7322-46f9-969e-c456de3d75b5 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.752014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c597c42-7dd9-4b94-ace0-197a14ed459f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:40.753463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c597c42-7dd9-4b94-ace0-197a14ed459f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-2c597c42-7dd9-4b94-ace0-197a14ed459f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:40.768164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:40.769303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 09:38:40.770468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:40.774284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 426/855] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:38:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:40.775532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17af5a-9353-4308-9422-891aae9e3eb3 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.775820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17af5a-9353-4308-9422-891aae9e3eb3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.776097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17af5a-9353-4308-9422-891aae9e3eb3 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.777401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 427/856] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:40.779510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17af5a-9353-4308-9422-891aae9e3eb3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.787032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defadca-e96f-4d9a-bba7-e6fcf3ea34bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.788066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defadca-e96f-4d9a-bba7-e6fcf3ea34bf None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.788510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defadca-e96f-4d9a-bba7-e6fcf3ea34bf None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.788933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defadca-e96f-4d9a-bba7-e6fcf3ea34bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.797800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defadca-e96f-4d9a-bba7-e6fcf3ea34bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:40.799391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defadca-e96f-4d9a-bba7-e6fcf3ea34bf None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8defadca-e96f-4d9a-bba7-e6fcf3ea34bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:40.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:40.816530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:40.817455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 430/857] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:38:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:40.818146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85814a55-3690-43e8-b444-47935d12958a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.818553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85814a55-3690-43e8-b444-47935d12958a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.818553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85814a55-3690-43e8-b444-47935d12958a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.835300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-85814a55-3690-43e8-b444-47935d12958a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:40.836480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-85814a55-3690-43e8-b444-47935d12958a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:40.840068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:40.886563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:40.903429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17af5a-9353-4308-9422-891aae9e3eb3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:40.910122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 431/858] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:40.910122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85814a55-3690-43e8-b444-47935d12958a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:40.944499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff54d58-b58d-463d-80f5-dcf0cf4e6132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:40.945167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff54d58-b58d-463d-80f5-dcf0cf4e6132 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:40.945448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff54d58-b58d-463d-80f5-dcf0cf4e6132 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:40.945724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff54d58-b58d-463d-80f5-dcf0cf4e6132 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:40.946588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff54d58-b58d-463d-80f5-dcf0cf4e6132 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:40.946894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff54d58-b58d-463d-80f5-dcf0cf4e6132 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:40.948323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff54d58-b58d-463d-80f5-dcf0cf4e6132 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:40.969761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 428/859] 10.222.0.42 () {56 vars in 1135 bytes} [Fri Jun 3 09:38:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:40.969761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff54d58-b58d-463d-80f5-dcf0cf4e6132 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:41.016774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85814a55-3690-43e8-b444-47935d12958a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:41.050700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a2181-f2db-4ac4-a278-9d8009870e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:41.051376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2181-f2db-4ac4-a278-9d8009870e02 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.051659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2181-f2db-4ac4-a278-9d8009870e02 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.051966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2181-f2db-4ac4-a278-9d8009870e02 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.052829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a2181-f2db-4ac4-a278-9d8009870e02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:41.053137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a2181-f2db-4ac4-a278-9d8009870e02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:41.054923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a2181-f2db-4ac4-a278-9d8009870e02 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:41.063792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:41.071904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff54d58-b58d-463d-80f5-dcf0cf4e6132 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:41.077486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 432/860] 10.222.0.42 () {56 vars in 1149 bytes} [Fri Jun 3 09:38:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:41.081310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a2181-f2db-4ac4-a278-9d8009870e02 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:41.108073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd1272-cade-47ca-99bd-d5be06a9bd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:41.108730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd1272-cade-47ca-99bd-d5be06a9bd7a None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.109008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd1272-cade-47ca-99bd-d5be06a9bd7a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.109280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd1272-cade-47ca-99bd-d5be06a9bd7a None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.110120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd1272-cade-47ca-99bd-d5be06a9bd7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:41.111565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd1272-cade-47ca-99bd-d5be06a9bd7a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:41.175136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a2181-f2db-4ac4-a278-9d8009870e02 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:41.185663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 429/861] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:38:40 2022] POST /identity/v3/services => generated 269 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:41.186696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd1272-cade-47ca-99bd-d5be06a9bd7a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:41.189221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:41.228365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d857c10-42e9-49d3-81e2-f5280af11135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:41.229321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d857c10-42e9-49d3-81e2-f5280af11135 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.229860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d857c10-42e9-49d3-81e2-f5280af11135 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.230345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d857c10-42e9-49d3-81e2-f5280af11135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.304491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:41.314826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d857c10-42e9-49d3-81e2-f5280af11135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:41.316433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d857c10-42e9-49d3-81e2-f5280af11135 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:41.337183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd1272-cade-47ca-99bd-d5be06a9bd7a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:41.378106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fadb7-8dd9-4870-92cd-193e3e092128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:41.378799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fadb7-8dd9-4870-92cd-193e3e092128 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.379084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fadb7-8dd9-4870-92cd-193e3e092128 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.379363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fadb7-8dd9-4870-92cd-193e3e092128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.443436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 433/862] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:41 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:38:41.444637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d857c10-42e9-49d3-81e2-f5280af11135 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:41.459108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fadb7-8dd9-4870-92cd-193e3e092128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:41.460565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fadb7-8dd9-4870-92cd-193e3e092128 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:41.462711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:41.567970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:41.577048 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d857c10-42e9-49d3-81e2-f5280af11135 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:41.580291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 430/863] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:41 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:38:41.586887 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fadb7-8dd9-4870-92cd-193e3e092128 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:41.625769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27c124-8e5d-4a7c-95e8-b96f6d10aa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:41.626596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27c124-8e5d-4a7c-95e8-b96f6d10aa95 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.626883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27c124-8e5d-4a7c-95e8-b96f6d10aa95 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.627163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27c124-8e5d-4a7c-95e8-b96f6d10aa95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.641712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27c124-8e5d-4a7c-95e8-b96f6d10aa95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:41.643544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27c124-8e5d-4a7c-95e8-b96f6d10aa95 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3b27c124-8e5d-4a7c-95e8-b96f6d10aa95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:41.669675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:41.670635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:41.671598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:41.671598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:41.671598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:41.671598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:41.671598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 434/864] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:41.676771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27c124-8e5d-4a7c-95e8-b96f6d10aa95 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:41.715804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fadb7-8dd9-4870-92cd-193e3e092128 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:41.715804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:41.762472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba669807-c664-47fa-8af5-b436121bb3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:41.763274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba669807-c664-47fa-8af5-b436121bb3c0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.763561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba669807-c664-47fa-8af5-b436121bb3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.763842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba669807-c664-47fa-8af5-b436121bb3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.776507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba669807-c664-47fa-8af5-b436121bb3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:41.778294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba669807-c664-47fa-8af5-b436121bb3c0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ba669807-c664-47fa-8af5-b436121bb3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:41.797763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:41.798837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:41.799697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:41.799697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:41.799697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:41.799697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:41.799918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27c124-8e5d-4a7c-95e8-b96f6d10aa95 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:41.802099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 431/865] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:41.807641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba669807-c664-47fa-8af5-b436121bb3c0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:41.843966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3186b-6bb9-456c-a05c-412f8b2b8af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:41.845485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3186b-6bb9-456c-a05c-412f8b2b8af3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.845772 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3186b-6bb9-456c-a05c-412f8b2b8af3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.846081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3186b-6bb9-456c-a05c-412f8b2b8af3 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.846818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3186b-6bb9-456c-a05c-412f8b2b8af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:41.847127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3186b-6bb9-456c-a05c-412f8b2b8af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:41.848566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3186b-6bb9-456c-a05c-412f8b2b8af3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:41.864556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 435/866] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:41.869454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcce44-a43c-48ff-9cdd-520a1b012ef9 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.869454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcce44-a43c-48ff-9cdd-520a1b012ef9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.869454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcce44-a43c-48ff-9cdd-520a1b012ef9 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.869454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 436/867] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:41.869796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcce44-a43c-48ff-9cdd-520a1b012ef9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:41.878860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:41.920341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba669807-c664-47fa-8af5-b436121bb3c0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:41.954103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac6174-0887-428d-9f55-03dde5456019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:41.954992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac6174-0887-428d-9f55-03dde5456019 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.955424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac6174-0887-428d-9f55-03dde5456019 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.955846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac6174-0887-428d-9f55-03dde5456019 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.956729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac6174-0887-428d-9f55-03dde5456019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:41.957180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac6174-0887-428d-9f55-03dde5456019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:41.959291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac6174-0887-428d-9f55-03dde5456019 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:41.973440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcce44-a43c-48ff-9cdd-520a1b012ef9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:41.975257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 432/868] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:41.976442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86917a9e-d6d7-4b77-8baa-0a474de8d8ad None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:41.976736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86917a9e-d6d7-4b77-8baa-0a474de8d8ad None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:41.977013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86917a9e-d6d7-4b77-8baa-0a474de8d8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:41.993036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-86917a9e-d6d7-4b77-8baa-0a474de8d8ad None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:41.994252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-86917a9e-d6d7-4b77-8baa-0a474de8d8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:42.003387 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:42.008946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704831e-0cad-4ef6-9110-89bda6de2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:42.009957 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704831e-0cad-4ef6-9110-89bda6de2b47 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:42.010414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704831e-0cad-4ef6-9110-89bda6de2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:42.010677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704831e-0cad-4ef6-9110-89bda6de2b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:42.017362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704831e-0cad-4ef6-9110-89bda6de2b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:42.018828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704831e-0cad-4ef6-9110-89bda6de2b47 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-9704831e-0cad-4ef6-9110-89bda6de2b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:42.033754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:42.034832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 437/869] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:42.041044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704831e-0cad-4ef6-9110-89bda6de2b47 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:42.071575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 433/870] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:41 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:42.072431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86917a9e-d6d7-4b77-8baa-0a474de8d8ad None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:42.113200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:42.150025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704831e-0cad-4ef6-9110-89bda6de2b47 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:42.185744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d316a3-9f8f-4b42-bc5e-853d5c8ee2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:42.186562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d316a3-9f8f-4b42-bc5e-853d5c8ee2cb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:42.186846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d316a3-9f8f-4b42-bc5e-853d5c8ee2cb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:42.187123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d316a3-9f8f-4b42-bc5e-853d5c8ee2cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:42.187400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86917a9e-d6d7-4b77-8baa-0a474de8d8ad None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:42.193661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d316a3-9f8f-4b42-bc5e-853d5c8ee2cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:42.195152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d316a3-9f8f-4b42-bc5e-853d5c8ee2cb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d9d316a3-9f8f-4b42-bc5e-853d5c8ee2cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:42.209053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:42.210202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 438/871] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:42.211102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d316a3-9f8f-4b42-bc5e-853d5c8ee2cb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:42.228718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed52df-8d86-4724-a37a-daf0004b7c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:42.229571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed52df-8d86-4724-a37a-daf0004b7c79 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:42.230046 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed52df-8d86-4724-a37a-daf0004b7c79 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:42.230548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed52df-8d86-4724-a37a-daf0004b7c79 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:42.231600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed52df-8d86-4724-a37a-daf0004b7c79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:42.232062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed52df-8d86-4724-a37a-daf0004b7c79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:42.233762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed52df-8d86-4724-a37a-daf0004b7c79 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:42.246648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 434/872] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:42.247472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed52df-8d86-4724-a37a-daf0004b7c79 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:42.253479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:42.318078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d316a3-9f8f-4b42-bc5e-853d5c8ee2cb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:42.344176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:42.353101 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b79c36-e5aa-4203-966c-4f02f7908db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:42.353769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b79c36-e5aa-4203-966c-4f02f7908db9 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:42.354079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b79c36-e5aa-4203-966c-4f02f7908db9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:42.354350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b79c36-e5aa-4203-966c-4f02f7908db9 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:42.355063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b79c36-e5aa-4203-966c-4f02f7908db9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:42.356386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed52df-8d86-4724-a37a-daf0004b7c79 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:42.356817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b79c36-e5aa-4203-966c-4f02f7908db9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:42.391334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152cae5-b303-474b-80b3-695d0574bf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:42.391921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152cae5-b303-474b-80b3-695d0574bf55 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:42.392210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152cae5-b303-474b-80b3-695d0574bf55 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:42.392488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152cae5-b303-474b-80b3-695d0574bf55 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:42.393269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152cae5-b303-474b-80b3-695d0574bf55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:42.393631 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152cae5-b303-474b-80b3-695d0574bf55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:42.395116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152cae5-b303-474b-80b3-695d0574bf55 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:42.409433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 435/873] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:42.410116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152cae5-b303-474b-80b3-695d0574bf55 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:42.459335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 439/874] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:38:42 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:42.460251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b79c36-e5aa-4203-966c-4f02f7908db9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:42.479942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:42.523109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152cae5-b303-474b-80b3-695d0574bf55 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:42.571685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220633e-1c9e-4fea-a25e-3d9c26ee95b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:42.572385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220633e-1c9e-4fea-a25e-3d9c26ee95b1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:42.572777 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220633e-1c9e-4fea-a25e-3d9c26ee95b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:42.572956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220633e-1c9e-4fea-a25e-3d9c26ee95b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:42.573846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220633e-1c9e-4fea-a25e-3d9c26ee95b1 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=37186dc0010247cd936f160b5eb0298b) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:42.574215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220633e-1c9e-4fea-a25e-3d9c26ee95b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:42.575688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220633e-1c9e-4fea-a25e-3d9c26ee95b1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:42.583194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 436/875] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:42 2022] GET /identity/v3/role_assignments?user.id=37186dc0010247cd936f160b5eb0298b&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=fe4d636f5c4842f18569347911e6cdec => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:42.610921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b79c36-e5aa-4203-966c-4f02f7908db9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:42.650821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd9a2a-75a2-429c-ab22-74fbf7eca2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:42.651774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd9a2a-75a2-429c-ab22-74fbf7eca2d2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:42.652218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd9a2a-75a2-429c-ab22-74fbf7eca2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:42.652654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd9a2a-75a2-429c-ab22-74fbf7eca2d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:42.653783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd9a2a-75a2-429c-ab22-74fbf7eca2d2 None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=fd85838dd97c4d74995302afe060388c) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:42.654285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd9a2a-75a2-429c-ab22-74fbf7eca2d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:42.655992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd9a2a-75a2-429c-ab22-74fbf7eca2d2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:42.664343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 440/876] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:42 2022] GET /identity/v3/role_assignments?user.id=fd85838dd97c4d74995302afe060388c&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:43.544165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2389f-bf09-468f-b5cf-6d8ee8fffbfe None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:43.544717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2389f-bf09-468f-b5cf-6d8ee8fffbfe None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:43.544717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2389f-bf09-468f-b5cf-6d8ee8fffbfe None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:43.546322 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 437/877] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:43.553118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17813280-023b-4577-87e9-5efa9a21822d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:43.553511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17813280-023b-4577-87e9-5efa9a21822d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:43.553673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17813280-023b-4577-87e9-5efa9a21822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:43.570780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-17813280-023b-4577-87e9-5efa9a21822d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:43.571944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-17813280-023b-4577-87e9-5efa9a21822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:43.619885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:43.641057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 441/878] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:43.663503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2389f-bf09-468f-b5cf-6d8ee8fffbfe None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:43.780519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2389f-bf09-468f-b5cf-6d8ee8fffbfe None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:43.807205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:43.818472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89bdfe-be2c-4fff-8b91-23dfe1b63353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:43.819320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89bdfe-be2c-4fff-8b91-23dfe1b63353 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:43.819609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89bdfe-be2c-4fff-8b91-23dfe1b63353 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:43.819889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89bdfe-be2c-4fff-8b91-23dfe1b63353 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:43.820547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89bdfe-be2c-4fff-8b91-23dfe1b63353 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:43.821938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89bdfe-be2c-4fff-8b91-23dfe1b63353 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-0c89bdfe-be2c-4fff-8b91-23dfe1b63353 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:43.835222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:43.836194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:43.837111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 438/879] 10.222.0.42 () {56 vars in 1132 bytes} [Fri Jun 3 09:38:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:43.842482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17813280-023b-4577-87e9-5efa9a21822d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:43.944599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17813280-023b-4577-87e9-5efa9a21822d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:43.980353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541f0e9-ebda-4cf8-8296-3846455df488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:43.981018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541f0e9-ebda-4cf8-8296-3846455df488 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:43.981298 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541f0e9-ebda-4cf8-8296-3846455df488 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:43.981571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541f0e9-ebda-4cf8-8296-3846455df488 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:43.982315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541f0e9-ebda-4cf8-8296-3846455df488 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:43.982618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541f0e9-ebda-4cf8-8296-3846455df488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:43.983980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541f0e9-ebda-4cf8-8296-3846455df488 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:43.989095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 442/880] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:43.994658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89bdfe-be2c-4fff-8b91-23dfe1b63353 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:44.100058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89bdfe-be2c-4fff-8b91-23dfe1b63353 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:44.135445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d786f3-4852-4509-8290-59e6f45c1221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:44.136116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d786f3-4852-4509-8290-59e6f45c1221 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:44.136407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d786f3-4852-4509-8290-59e6f45c1221 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:44.136689 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d786f3-4852-4509-8290-59e6f45c1221 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:44.137399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d786f3-4852-4509-8290-59e6f45c1221 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:44.137707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d786f3-4852-4509-8290-59e6f45c1221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:44.139121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d786f3-4852-4509-8290-59e6f45c1221 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:44.144667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 439/881] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:44.150501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541f0e9-ebda-4cf8-8296-3846455df488 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:44.197700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:44.253096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541f0e9-ebda-4cf8-8296-3846455df488 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:44.289496 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213642c-78f7-4e83-96de-f504227e189f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:44.290434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213642c-78f7-4e83-96de-f504227e189f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:44.290893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213642c-78f7-4e83-96de-f504227e189f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:44.291336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213642c-78f7-4e83-96de-f504227e189f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:44.292267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213642c-78f7-4e83-96de-f504227e189f None admin] RBAC: Adding query filter params (interface=public, service_id=63c4574bc195450d955b7558b3cf0131, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:44.292729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213642c-78f7-4e83-96de-f504227e189f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:44.294265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213642c-78f7-4e83-96de-f504227e189f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:44.299996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 443/882] 10.222.0.42 () {56 vars in 1251 bytes} [Fri Jun 3 09:38:44 2022] GET /identity/v3/endpoints?service_id=63c4574bc195450d955b7558b3cf0131&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:44.308503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d786f3-4852-4509-8290-59e6f45c1221 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:44.362290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:44.416607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d786f3-4852-4509-8290-59e6f45c1221 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:44.452330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e76bd-7833-452c-a50a-957a1b010e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:44.453004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e76bd-7833-452c-a50a-957a1b010e95 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:44.453296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e76bd-7833-452c-a50a-957a1b010e95 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:44.453575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e76bd-7833-452c-a50a-957a1b010e95 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:44.454295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e76bd-7833-452c-a50a-957a1b010e95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:44.455678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e76bd-7833-452c-a50a-957a1b010e95 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:44.462102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 440/883] 10.222.0.42 () {56 vars in 1087 bytes} [Fri Jun 3 09:38:44 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:38:44.830348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec0902-92f3-4e10-8390-aabc7e714c9d None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:44.830872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec0902-92f3-4e10-8390-aabc7e714c9d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:44.830872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec0902-92f3-4e10-8390-aabc7e714c9d None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:44.832536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 444/884] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:44.838473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03741e-d95b-47e5-bfdf-0c4f6fadbfa6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:44.838947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03741e-d95b-47e5-bfdf-0c4f6fadbfa6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:44.839025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03741e-d95b-47e5-bfdf-0c4f6fadbfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:44.856406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1e03741e-d95b-47e5-bfdf-0c4f6fadbfa6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:44.857608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1e03741e-d95b-47e5-bfdf-0c4f6fadbfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:44.929995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 441/885] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:44.948691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec0902-92f3-4e10-8390-aabc7e714c9d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:44.978595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7640f8-c46c-4659-a462-b70b2d6beda6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:44.979084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7640f8-c46c-4659-a462-b70b2d6beda6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:44.979375 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7640f8-c46c-4659-a462-b70b2d6beda6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:44.981598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 442/886] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:38:44.986635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c3631-4707-44fd-af59-b29136753d88 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:44.986928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c3631-4707-44fd-af59-b29136753d88 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:44.987213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c3631-4707-44fd-af59-b29136753d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.004584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-413c3631-4707-44fd-af59-b29136753d88 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:45.007072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-413c3631-4707-44fd-af59-b29136753d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:45.053542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec0902-92f3-4e10-8390-aabc7e714c9d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.062237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:45.063128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:45.089916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5275d-cd7a-451d-8078-d7622d01472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.090839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5275d-cd7a-451d-8078-d7622d01472b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.091249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5275d-cd7a-451d-8078-d7622d01472b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.091665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5275d-cd7a-451d-8078-d7622d01472b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.092410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5275d-cd7a-451d-8078-d7622d01472b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:45.094011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5275d-cd7a-451d-8078-d7622d01472b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:45.104514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 443/887] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-c7f5275d-cd7a-451d-8078-d7622d01472b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:45.108302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:45.109393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:45.116064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 445/888] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:38:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:45.118001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c3631-4707-44fd-af59-b29136753d88 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.139488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5275d-cd7a-451d-8078-d7622d01472b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.254094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c3631-4707-44fd-af59-b29136753d88 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.258277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5275d-cd7a-451d-8078-d7622d01472b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.291546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ee94e-eac7-4f5b-b407-b331d133b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.292218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ee94e-eac7-4f5b-b407-b331d133b8eb None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.292504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ee94e-eac7-4f5b-b407-b331d133b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.292786 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ee94e-eac7-4f5b-b407-b331d133b8eb None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.293371 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b6133-b336-49be-96c9-0ea567a0f698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.293484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ee94e-eac7-4f5b-b407-b331d133b8eb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:45.293794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ee94e-eac7-4f5b-b407-b331d133b8eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:45.294329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b6133-b336-49be-96c9-0ea567a0f698 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.294618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b6133-b336-49be-96c9-0ea567a0f698 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.294898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b6133-b336-49be-96c9-0ea567a0f698 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.295257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ee94e-eac7-4f5b-b407-b331d133b8eb None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:45.295546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b6133-b336-49be-96c9-0ea567a0f698 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:45.296919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b6133-b336-49be-96c9-0ea567a0f698 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:45.305573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 444/889] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:45.305573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d8ca0-da56-43e6-9883-f1af72998af8 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.305573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d8ca0-da56-43e6-9883-f1af72998af8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.305573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d8ca0-da56-43e6-9883-f1af72998af8 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.305573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 445/890] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:45.311015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d8ca0-da56-43e6-9883-f1af72998af8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d74b6133-b336-49be-96c9-0ea567a0f698 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:45.315215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:45.316232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:45.317080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 446/891] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:45.318483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df29f6-31a6-4ead-90ea-bc740b39d30a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.318483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df29f6-31a6-4ead-90ea-bc740b39d30a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.324190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df29f6-31a6-4ead-90ea-bc740b39d30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.342542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c1df29f6-31a6-4ead-90ea-bc740b39d30a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:45.344213 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c1df29f6-31a6-4ead-90ea-bc740b39d30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:45.358554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:45.373955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:45.413536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 447/892] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:45.414973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df29f6-31a6-4ead-90ea-bc740b39d30a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.431032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d8ca0-da56-43e6-9883-f1af72998af8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.471288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503439e-b116-4da4-b02a-f44ca33ef681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.472075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503439e-b116-4da4-b02a-f44ca33ef681 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.472360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503439e-b116-4da4-b02a-f44ca33ef681 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.472694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503439e-b116-4da4-b02a-f44ca33ef681 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.474110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503439e-b116-4da4-b02a-f44ca33ef681 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:45.474445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503439e-b116-4da4-b02a-f44ca33ef681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:45.475803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503439e-b116-4da4-b02a-f44ca33ef681 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:45.480566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 446/893] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:45.482804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503439e-b116-4da4-b02a-f44ca33ef681 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.519436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df29f6-31a6-4ead-90ea-bc740b39d30a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.556850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb712099-213f-4a81-870c-68af7e801504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.557774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb712099-213f-4a81-870c-68af7e801504 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.558230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb712099-213f-4a81-870c-68af7e801504 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.558651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb712099-213f-4a81-870c-68af7e801504 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.559494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb712099-213f-4a81-870c-68af7e801504 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:45.560034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb712099-213f-4a81-870c-68af7e801504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:45.561686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb712099-213f-4a81-870c-68af7e801504 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:45.567061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 448/894] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:45.567881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb712099-213f-4a81-870c-68af7e801504 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.588531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503439e-b116-4da4-b02a-f44ca33ef681 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.590828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:45.599719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:45.627769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a4f1d-dee5-4b1b-ac94-99d9cbf603cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.628594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a4f1d-dee5-4b1b-ac94-99d9cbf603cc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.628893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a4f1d-dee5-4b1b-ac94-99d9cbf603cc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.629147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a4f1d-dee5-4b1b-ac94-99d9cbf603cc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.641152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a4f1d-dee5-4b1b-ac94-99d9cbf603cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-657a4f1d-dee5-4b1b-ac94-99d9cbf603cc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:45.649351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:45.650296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:45.651139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 447/895] 10.222.0.42 () {56 vars in 1120 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:45.651817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a4f1d-dee5-4b1b-ac94-99d9cbf603cc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.679163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb712099-213f-4a81-870c-68af7e801504 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.721490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d07487-fe33-4c9a-98a9-ad903307035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.722190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d07487-fe33-4c9a-98a9-ad903307035c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.722477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d07487-fe33-4c9a-98a9-ad903307035c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.722758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d07487-fe33-4c9a-98a9-ad903307035c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.723557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d07487-fe33-4c9a-98a9-ad903307035c None admin] RBAC: Adding query filter params (interface=public, service_id=cbe40b4a78284715b89270477052a09b, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:45.723904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d07487-fe33-4c9a-98a9-ad903307035c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:45.725281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d07487-fe33-4c9a-98a9-ad903307035c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:45.730413 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 449/896] 10.222.0.42 () {56 vars in 1251 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/endpoints?service_id=cbe40b4a78284715b89270477052a09b&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:45.731189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d07487-fe33-4c9a-98a9-ad903307035c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.771019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a4f1d-dee5-4b1b-ac94-99d9cbf603cc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.805240 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b967e9-c5a1-48e8-801d-6ff55d799116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.805927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b967e9-c5a1-48e8-801d-6ff55d799116 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.806214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b967e9-c5a1-48e8-801d-6ff55d799116 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.806487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b967e9-c5a1-48e8-801d-6ff55d799116 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.807183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b967e9-c5a1-48e8-801d-6ff55d799116 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:45.807484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b967e9-c5a1-48e8-801d-6ff55d799116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:45.808841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b967e9-c5a1-48e8-801d-6ff55d799116 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:45.813189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 448/897] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:45.813975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b967e9-c5a1-48e8-801d-6ff55d799116 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.848087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:45.853031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:45.863034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d07487-fe33-4c9a-98a9-ad903307035c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.914697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcded93-65f3-4c20-ab06-4f6184ec3f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.916151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcded93-65f3-4c20-ab06-4f6184ec3f7c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.916758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcded93-65f3-4c20-ab06-4f6184ec3f7c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.917285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcded93-65f3-4c20-ab06-4f6184ec3f7c None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.918510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcded93-65f3-4c20-ab06-4f6184ec3f7c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:45.918993 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcded93-65f3-4c20-ab06-4f6184ec3f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:45.921265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcded93-65f3-4c20-ab06-4f6184ec3f7c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:45.938955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b967e9-c5a1-48e8-801d-6ff55d799116 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:45.939899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 450/898] 10.222.0.42 () {56 vars in 1115 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:45.942557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcded93-65f3-4c20-ab06-4f6184ec3f7c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:45.984596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839be71d-e6e1-4664-aef8-8f9fac614d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:45.985578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839be71d-e6e1-4664-aef8-8f9fac614d36 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:45.986065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839be71d-e6e1-4664-aef8-8f9fac614d36 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:45.986788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839be71d-e6e1-4664-aef8-8f9fac614d36 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:45.988626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839be71d-e6e1-4664-aef8-8f9fac614d36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:45.990815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839be71d-e6e1-4664-aef8-8f9fac614d36 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:45.999754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 449/899] 10.222.0.42 () {56 vars in 1087 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:38:46.074034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcded93-65f3-4c20-ab06-4f6184ec3f7c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:46.112493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f865f-483f-43ef-aa13-31a766d639cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:46.113316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f865f-483f-43ef-aa13-31a766d639cc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:46.113605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f865f-483f-43ef-aa13-31a766d639cc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:46.113909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f865f-483f-43ef-aa13-31a766d639cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:46.126746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f865f-483f-43ef-aa13-31a766d639cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:46.128235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f865f-483f-43ef-aa13-31a766d639cc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-dd5f865f-483f-43ef-aa13-31a766d639cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:46.147776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:46.148841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:46.149730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:46.149730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:46.149730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:46.149730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:46.153986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 451/900] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:46.155403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839be71d-e6e1-4664-aef8-8f9fac614d36 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:46.256052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:46.264883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839be71d-e6e1-4664-aef8-8f9fac614d36 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:46.301031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece080e-f11b-4dff-ba7a-2cb91e864457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:46.301732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece080e-f11b-4dff-ba7a-2cb91e864457 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:46.302056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece080e-f11b-4dff-ba7a-2cb91e864457 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:46.302339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece080e-f11b-4dff-ba7a-2cb91e864457 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:46.303067 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece080e-f11b-4dff-ba7a-2cb91e864457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:46.303378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece080e-f11b-4dff-ba7a-2cb91e864457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:46.304778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece080e-f11b-4dff-ba7a-2cb91e864457 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:46.321707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 450/901] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:46.323104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f865f-483f-43ef-aa13-31a766d639cc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:46.335672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:46.435673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f865f-483f-43ef-aa13-31a766d639cc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:46.473688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb865f6-aa0a-4feb-88a8-f6334d7d2bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:46.474541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb865f6-aa0a-4feb-88a8-f6334d7d2bed None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:46.474821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb865f6-aa0a-4feb-88a8-f6334d7d2bed None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:46.475094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb865f6-aa0a-4feb-88a8-f6334d7d2bed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:46.483848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb865f6-aa0a-4feb-88a8-f6334d7d2bed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:46.485319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb865f6-aa0a-4feb-88a8-f6334d7d2bed None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-4bb865f6-aa0a-4feb-88a8-f6334d7d2bed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:46.500056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:46.501057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:46.502454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 452/902] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:38:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:46.509792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece080e-f11b-4dff-ba7a-2cb91e864457 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:46.617895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece080e-f11b-4dff-ba7a-2cb91e864457 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:46.653517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dea97-bad0-4893-b669-493d1f008e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:46.654608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dea97-bad0-4893-b669-493d1f008e08 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:46.655101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dea97-bad0-4893-b669-493d1f008e08 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:46.655579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dea97-bad0-4893-b669-493d1f008e08 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:46.656637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dea97-bad0-4893-b669-493d1f008e08 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:46.657191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dea97-bad0-4893-b669-493d1f008e08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:46.658849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dea97-bad0-4893-b669-493d1f008e08 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:46.681515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 451/903] 10.222.0.42 () {56 vars in 1149 bytes} [Fri Jun 3 09:38:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:46.687144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb865f6-aa0a-4feb-88a8-f6334d7d2bed None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:46.766666 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b625e-7502-4912-8769-89a53f4a192b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:46.766994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b625e-7502-4912-8769-89a53f4a192b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:46.767265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b625e-7502-4912-8769-89a53f4a192b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:46.768892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 452/904] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:38:46.771694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:46.774537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651ceed-f6bc-4266-8b0a-ebdc5c8d042c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:46.774820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651ceed-f6bc-4266-8b0a-ebdc5c8d042c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:46.775100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651ceed-f6bc-4266-8b0a-ebdc5c8d042c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:46.792342 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4651ceed-f6bc-4266-8b0a-ebdc5c8d042c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:46.793512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4651ceed-f6bc-4266-8b0a-ebdc5c8d042c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:46.808917 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb865f6-aa0a-4feb-88a8-f6334d7d2bed None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:46.826255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:46.847928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149c28-0849-40da-a813-4fb143779e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:46.849500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149c28-0849-40da-a813-4fb143779e08 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:46.852588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149c28-0849-40da-a813-4fb143779e08 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:46.852588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149c28-0849-40da-a813-4fb143779e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:46.873989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 453/905] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:38:46.892977 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651ceed-f6bc-4266-8b0a-ebdc5c8d042c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:46.925726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf149c28-0849-40da-a813-4fb143779e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:46.927270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf149c28-0849-40da-a813-4fb143779e08 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:46.995785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:46.998983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651ceed-f6bc-4266-8b0a-ebdc5c8d042c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:47.037737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e8f04-695b-4947-83c8-e39b1ecc28bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:47.038600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e8f04-695b-4947-83c8-e39b1ecc28bf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:47.039138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e8f04-695b-4947-83c8-e39b1ecc28bf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:47.039415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e8f04-695b-4947-83c8-e39b1ecc28bf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:47.040070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e8f04-695b-4947-83c8-e39b1ecc28bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:47.041750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e8f04-695b-4947-83c8-e39b1ecc28bf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:47.042047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 453/906] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:46 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:38:47.053917 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149c28-0849-40da-a813-4fb143779e08 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-473e8f04-695b-4947-83c8-e39b1ecc28bf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:47.056614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:47.057519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 09:38:47.058383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:47.062582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 454/907] 10.222.0.42 () {56 vars in 1132 bytes} [Fri Jun 3 09:38:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:47.064242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e8f04-695b-4947-83c8-e39b1ecc28bf None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:47.071082 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:47.182846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149c28-0849-40da-a813-4fb143779e08 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:47.191449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e8f04-695b-4947-83c8-e39b1ecc28bf None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:47.220775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19b26-a1f4-465f-92bc-cfe48dddc354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:47.222051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19b26-a1f4-465f-92bc-cfe48dddc354 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:47.222531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19b26-a1f4-465f-92bc-cfe48dddc354 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:47.223021 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f19b26-a1f4-465f-92bc-cfe48dddc354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:47.227207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a759929-cfc3-4adc-bd77-d00b5d2e0122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:47.227881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a759929-cfc3-4adc-bd77-d00b5d2e0122 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:47.228216 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a759929-cfc3-4adc-bd77-d00b5d2e0122 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:47.228494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a759929-cfc3-4adc-bd77-d00b5d2e0122 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:47.229259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a759929-cfc3-4adc-bd77-d00b5d2e0122 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:47.229569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a759929-cfc3-4adc-bd77-d00b5d2e0122 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:47.230979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a759929-cfc3-4adc-bd77-d00b5d2e0122 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:47.236788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f19b26-a1f4-465f-92bc-cfe48dddc354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:47.238578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f19b26-a1f4-465f-92bc-cfe48dddc354 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:47.241529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 455/908] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:47.242563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a759929-cfc3-4adc-bd77-d00b5d2e0122 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f2f19b26-a1f4-465f-92bc-cfe48dddc354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:47.260645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:47.261549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:47.262785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:47.262785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:47.262785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:47.262785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:47.264824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 454/909] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:47.273931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19b26-a1f4-465f-92bc-cfe48dddc354 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:47.297538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:47.318800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:47.354297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a759929-cfc3-4adc-bd77-d00b5d2e0122 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:47.390323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cddda-e69b-4c92-8df0-1cc6474ac615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:47.390988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cddda-e69b-4c92-8df0-1cc6474ac615 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:47.391281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cddda-e69b-4c92-8df0-1cc6474ac615 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:47.391549 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cddda-e69b-4c92-8df0-1cc6474ac615 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:47.392255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cddda-e69b-4c92-8df0-1cc6474ac615 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:47.392562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cddda-e69b-4c92-8df0-1cc6474ac615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:47.394232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cddda-e69b-4c92-8df0-1cc6474ac615 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:47.398312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f19b26-a1f4-465f-92bc-cfe48dddc354 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:47.404169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 456/910] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:47.404996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cddda-e69b-4c92-8df0-1cc6474ac615 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:47.444269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122e6f0-e5ab-4513-ad9b-7a1f2c2adcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:47.445273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122e6f0-e5ab-4513-ad9b-7a1f2c2adcd8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:47.445763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122e6f0-e5ab-4513-ad9b-7a1f2c2adcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:47.446269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122e6f0-e5ab-4513-ad9b-7a1f2c2adcd8 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:47.447613 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122e6f0-e5ab-4513-ad9b-7a1f2c2adcd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:47.448131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122e6f0-e5ab-4513-ad9b-7a1f2c2adcd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:47.450160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122e6f0-e5ab-4513-ad9b-7a1f2c2adcd8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:47.466391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 455/911] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:47.471670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122e6f0-e5ab-4513-ad9b-7a1f2c2adcd8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:47.514668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cddda-e69b-4c92-8df0-1cc6474ac615 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:47.545701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:47.551365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa8b70-76cf-4170-b905-b1d547163021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:47.552561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa8b70-76cf-4170-b905-b1d547163021 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:47.552845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa8b70-76cf-4170-b905-b1d547163021 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:47.553177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa8b70-76cf-4170-b905-b1d547163021 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:47.554124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa8b70-76cf-4170-b905-b1d547163021 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:47.555611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa8b70-76cf-4170-b905-b1d547163021 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:47.574568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:47.594528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122e6f0-e5ab-4513-ad9b-7a1f2c2adcd8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:47.629437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdbffb-7230-448d-8bab-dc120844d46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:47.630366 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdbffb-7230-448d-8bab-dc120844d46d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:47.630653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdbffb-7230-448d-8bab-dc120844d46d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:47.630931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdbffb-7230-448d-8bab-dc120844d46d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:47.637718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdbffb-7230-448d-8bab-dc120844d46d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:47.639269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fdbffb-7230-448d-8bab-dc120844d46d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:47.654039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 457/912] 10.222.0.42 () {60 vars in 1144 bytes} [Fri Jun 3 09:38:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-37fdbffb-7230-448d-8bab-dc120844d46d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:47.654533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:47.663204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:47.664383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:47.664383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:47.664383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:47.664383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:47.664383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:47.664383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:47.664383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:47.664383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:47.664383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 456/913] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:47.664856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa8b70-76cf-4170-b905-b1d547163021 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:47.797474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa8b70-76cf-4170-b905-b1d547163021 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:47.835816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a76d3-b776-43f8-87bb-443bc0ab5e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:47.836501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a76d3-b776-43f8-87bb-443bc0ab5e5f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:47.836789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a76d3-b776-43f8-87bb-443bc0ab5e5f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:47.837128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a76d3-b776-43f8-87bb-443bc0ab5e5f None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:47.838029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a76d3-b776-43f8-87bb-443bc0ab5e5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:47.838472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a76d3-b776-43f8-87bb-443bc0ab5e5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:47.839933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a76d3-b776-43f8-87bb-443bc0ab5e5f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:47.858066 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 458/914] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:47.860702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdbffb-7230-448d-8bab-dc120844d46d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:47.975687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdbffb-7230-448d-8bab-dc120844d46d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:48.013394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b0c0c-06ca-4c01-b56e-f8c17eeb4584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:48.014104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0c0c-06ca-4c01-b56e-f8c17eeb4584 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:48.014394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0c0c-06ca-4c01-b56e-f8c17eeb4584 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:48.014676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b0c0c-06ca-4c01-b56e-f8c17eeb4584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:48.015511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b0c0c-06ca-4c01-b56e-f8c17eeb4584 None admin] RBAC: Adding query filter params (role.id=660489b44e624266a7f8293c906652ed, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=fd85838dd97c4d74995302afe060388c) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:48.015886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b0c0c-06ca-4c01-b56e-f8c17eeb4584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:48.017339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b0c0c-06ca-4c01-b56e-f8c17eeb4584 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:48.030008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 457/915] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:47 2022] GET /identity/v3/role_assignments?user.id=fd85838dd97c4d74995302afe060388c&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=660489b44e624266a7f8293c906652ed => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:48.269351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a8091-07c2-4c59-82f9-afd8372038e3 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:48.269833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a8091-07c2-4c59-82f9-afd8372038e3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:48.269833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a8091-07c2-4c59-82f9-afd8372038e3 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:48.272129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 459/916] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:48.277645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433819b-a07f-4539-a9a3-1f308e486c5a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:48.277957 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433819b-a07f-4539-a9a3-1f308e486c5a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:48.278250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433819b-a07f-4539-a9a3-1f308e486c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:48.287190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:48.298180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9433819b-a07f-4539-a9a3-1f308e486c5a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:48.299373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9433819b-a07f-4539-a9a3-1f308e486c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:48.305924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dbe19-4e0d-4add-a92f-444881df3703 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:48.305924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dbe19-4e0d-4add-a92f-444881df3703 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:48.305924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dbe19-4e0d-4add-a92f-444881df3703 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:48.307349 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 460/917] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:38:48.312503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e3f6-cfdc-41e4-8f12-7acf155633cc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:48.312807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e3f6-cfdc-41e4-8f12-7acf155633cc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:48.313077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e3f6-cfdc-41e4-8f12-7acf155633cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:48.335093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3e53e3f6-cfdc-41e4-8f12-7acf155633cc None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:48.336705 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3e53e3f6-cfdc-41e4-8f12-7acf155633cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:48.374596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 458/918] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:48.391053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:48.397847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433819b-a07f-4539-a9a3-1f308e486c5a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:48.425369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 461/919] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:38:48.448621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53e3f6-cfdc-41e4-8f12-7acf155633cc None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:48.524115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433819b-a07f-4539-a9a3-1f308e486c5a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:48.561632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53e3f6-cfdc-41e4-8f12-7acf155633cc None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:48.575079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a57c01-4a46-4618-bb4b-515f6b30595d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:48.576239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a57c01-4a46-4618-bb4b-515f6b30595d None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:48.576681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a57c01-4a46-4618-bb4b-515f6b30595d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:48.577355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a57c01-4a46-4618-bb4b-515f6b30595d None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:48.578778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a57c01-4a46-4618-bb4b-515f6b30595d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:48.580832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a57c01-4a46-4618-bb4b-515f6b30595d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:48.586212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:48.601504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60b784-e171-4a64-b801-3e35f513c6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:48.602357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60b784-e171-4a64-b801-3e35f513c6c9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:48.602640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60b784-e171-4a64-b801-3e35f513c6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:48.602915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60b784-e171-4a64-b801-3e35f513c6c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:48.603569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60b784-e171-4a64-b801-3e35f513c6c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:48.604942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60b784-e171-4a64-b801-3e35f513c6c9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-4c60b784-e171-4a64-b801-3e35f513c6c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:48.618020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:48.619586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:48.620911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 462/920] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:38:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:48.626800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60b784-e171-4a64-b801-3e35f513c6c9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:48.652158 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 459/921] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:38:48 2022] POST /identity/v3/services => generated 241 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:38:48.678641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:48.745159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60b784-e171-4a64-b801-3e35f513c6c9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:48.789864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1823578-fc6c-4923-80a0-c6a2e7e7964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:48.790554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1823578-fc6c-4923-80a0-c6a2e7e7964d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:48.790834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1823578-fc6c-4923-80a0-c6a2e7e7964d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:48.791107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1823578-fc6c-4923-80a0-c6a2e7e7964d None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:48.791813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1823578-fc6c-4923-80a0-c6a2e7e7964d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:48.792117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1823578-fc6c-4923-80a0-c6a2e7e7964d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:48.793505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1823578-fc6c-4923-80a0-c6a2e7e7964d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:48.799390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 463/922] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:38:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:48.807045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a57c01-4a46-4618-bb4b-515f6b30595d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:48.919318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a57c01-4a46-4618-bb4b-515f6b30595d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:48.953914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49e010-b7e8-4274-a69c-362ef65a27af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:48.954601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49e010-b7e8-4274-a69c-362ef65a27af None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:48.954882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49e010-b7e8-4274-a69c-362ef65a27af None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:48.955319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49e010-b7e8-4274-a69c-362ef65a27af None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:48.956030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49e010-b7e8-4274-a69c-362ef65a27af None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:48.956334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49e010-b7e8-4274-a69c-362ef65a27af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:48.957704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49e010-b7e8-4274-a69c-362ef65a27af None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:48.964882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 460/923] 10.222.0.42 () {56 vars in 1114 bytes} [Fri Jun 3 09:38:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:48.970446 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1823578-fc6c-4923-80a0-c6a2e7e7964d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:49.071943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1823578-fc6c-4923-80a0-c6a2e7e7964d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:49.082299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:49.107499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91694ca3-6eb3-44e1-b52b-8a0872bb92f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:49.108153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91694ca3-6eb3-44e1-b52b-8a0872bb92f8 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:49.108434 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91694ca3-6eb3-44e1-b52b-8a0872bb92f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:49.108709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91694ca3-6eb3-44e1-b52b-8a0872bb92f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:49.109457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91694ca3-6eb3-44e1-b52b-8a0872bb92f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:49.110893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91694ca3-6eb3-44e1-b52b-8a0872bb92f8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:49.216348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 464/924] 10.222.0.42 () {60 vars in 1144 bytes} [Fri Jun 3 09:38:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:50.047277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb9d6-380f-4eee-a127-d564f9620714 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:50.047776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb9d6-380f-4eee-a127-d564f9620714 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:50.047852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdb9d6-380f-4eee-a127-d564f9620714 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:50.049417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 461/925] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:50.054895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a955ad-ee6c-444e-ba65-d88aaa75bdbe None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:50.055397 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a955ad-ee6c-444e-ba65-d88aaa75bdbe None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:50.055591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a955ad-ee6c-444e-ba65-d88aaa75bdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:50.077297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-17a955ad-ee6c-444e-ba65-d88aaa75bdbe None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:50.078500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-17a955ad-ee6c-444e-ba65-d88aaa75bdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:50.153118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 465/926] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:50.174586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdb9d6-380f-4eee-a127-d564f9620714 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:50.235001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:50.277825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdb9d6-380f-4eee-a127-d564f9620714 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:50.312193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a50b7-f28e-4419-b25f-dd4fd0203220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:50.312909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a50b7-f28e-4419-b25f-dd4fd0203220 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:50.313192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a50b7-f28e-4419-b25f-dd4fd0203220 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:50.313468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a50b7-f28e-4419-b25f-dd4fd0203220 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:50.314143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a50b7-f28e-4419-b25f-dd4fd0203220 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:50.315507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a50b7-f28e-4419-b25f-dd4fd0203220 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-731a50b7-f28e-4419-b25f-dd4fd0203220 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:50.328175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:50.329190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:50.330488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 462/927] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:50.335478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a50b7-f28e-4419-b25f-dd4fd0203220 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:50.335758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d682d-0540-4fe3-9ba3-5c57d37dc38f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:50.336047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d682d-0540-4fe3-9ba3-5c57d37dc38f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:50.336329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d682d-0540-4fe3-9ba3-5c57d37dc38f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:50.337696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 466/928] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:50.344146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0a3fe-46d7-4c41-9f46-8e6851afaacd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:50.344515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0a3fe-46d7-4c41-9f46-8e6851afaacd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:50.344869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0a3fe-46d7-4c41-9f46-8e6851afaacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:50.366399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-cfe0a3fe-46d7-4c41-9f46-8e6851afaacd None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:50.368416 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cfe0a3fe-46d7-4c41-9f46-8e6851afaacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:50.452804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a50b7-f28e-4419-b25f-dd4fd0203220 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:50.461973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 467/929] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:38:50.478251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:50.486034 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0a3fe-46d7-4c41-9f46-8e6851afaacd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:50.495759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c776025-3327-475e-b1a0-d39a6786e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:50.496437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c776025-3327-475e-b1a0-d39a6786e032 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:50.496721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c776025-3327-475e-b1a0-d39a6786e032 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:50.497466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c776025-3327-475e-b1a0-d39a6786e032 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:50.498217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c776025-3327-475e-b1a0-d39a6786e032 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:50.499101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:50.499562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c776025-3327-475e-b1a0-d39a6786e032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:50.501232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c776025-3327-475e-b1a0-d39a6786e032 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:50.509993 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 463/930] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:50.513185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c776025-3327-475e-b1a0-d39a6786e032 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:50.602022 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0a3fe-46d7-4c41-9f46-8e6851afaacd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:50.645797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829193b-db9f-4439-bc4d-608861a3bd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:50.646741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829193b-db9f-4439-bc4d-608861a3bd14 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:50.646984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829193b-db9f-4439-bc4d-608861a3bd14 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:50.647620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829193b-db9f-4439-bc4d-608861a3bd14 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:50.649946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c776025-3327-475e-b1a0-d39a6786e032 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:50.659704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829193b-db9f-4439-bc4d-608861a3bd14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-5829193b-db9f-4439-bc4d-608861a3bd14 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:50.669318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:50.670509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:38:50.671536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:50.678634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 468/931] 10.222.0.42 () {56 vars in 1120 bytes} [Fri Jun 3 09:38:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:50.681545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829193b-db9f-4439-bc4d-608861a3bd14 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:50.699411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481b261-765c-4abf-a77a-b09032a97d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:50.700343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481b261-765c-4abf-a77a-b09032a97d77 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:50.700723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481b261-765c-4abf-a77a-b09032a97d77 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:50.701132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481b261-765c-4abf-a77a-b09032a97d77 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:50.702191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481b261-765c-4abf-a77a-b09032a97d77 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:50.702547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481b261-765c-4abf-a77a-b09032a97d77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:50.703987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481b261-765c-4abf-a77a-b09032a97d77 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:50.717919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 464/932] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:50.782939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:50.821541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829193b-db9f-4439-bc4d-608861a3bd14 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:50.860432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ffb3c-8c3e-47c2-bef8-59c3e7610d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:50.861483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ffb3c-8c3e-47c2-bef8-59c3e7610d11 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:50.862004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ffb3c-8c3e-47c2-bef8-59c3e7610d11 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:50.862485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ffb3c-8c3e-47c2-bef8-59c3e7610d11 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:50.863523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ffb3c-8c3e-47c2-bef8-59c3e7610d11 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:50.864116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ffb3c-8c3e-47c2-bef8-59c3e7610d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:50.865761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ffb3c-8c3e-47c2-bef8-59c3e7610d11 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:50.876597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 469/933] 10.222.0.42 () {56 vars in 1115 bytes} [Fri Jun 3 09:38:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:50.883319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481b261-765c-4abf-a77a-b09032a97d77 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:50.939479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:50.990570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481b261-765c-4abf-a77a-b09032a97d77 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.027561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08a824-2727-44a3-8b9c-59044a3f7437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.028360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08a824-2727-44a3-8b9c-59044a3f7437 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.028643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08a824-2727-44a3-8b9c-59044a3f7437 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.028923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08a824-2727-44a3-8b9c-59044a3f7437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.040402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08a824-2727-44a3-8b9c-59044a3f7437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.041971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08a824-2727-44a3-8b9c-59044a3f7437 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8d08a824-2727-44a3-8b9c-59044a3f7437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:51.061436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:51.062815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:51.063686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:51.063686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:51.063686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:51.063686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:51.063912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507070c-e5ec-4906-93f3-90846ea115b9 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.063912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507070c-e5ec-4906-93f3-90846ea115b9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.063912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507070c-e5ec-4906-93f3-90846ea115b9 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.064364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 465/934] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:51.064506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 470/935] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:38:51.069574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507070c-e5ec-4906-93f3-90846ea115b9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.070081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16f60e-1fed-4955-81ab-684dfd071e86 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.070371 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16f60e-1fed-4955-81ab-684dfd071e86 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.070652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16f60e-1fed-4955-81ab-684dfd071e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.088082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8f16f60e-1fed-4955-81ab-684dfd071e86 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:51.089265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8f16f60e-1fed-4955-81ab-684dfd071e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:51.171872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 466/936] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:51.190883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507070c-e5ec-4906-93f3-90846ea115b9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.195295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16f60e-1fed-4955-81ab-684dfd071e86 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.226949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:51.232472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e93e29-018b-43e5-bfa1-445d3d7e8bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.233218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e93e29-018b-43e5-bfa1-445d3d7e8bd4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.233500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e93e29-018b-43e5-bfa1-445d3d7e8bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.233807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e93e29-018b-43e5-bfa1-445d3d7e8bd4 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.234648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e93e29-018b-43e5-bfa1-445d3d7e8bd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:51.240533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e93e29-018b-43e5-bfa1-445d3d7e8bd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.240653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:51.240736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e93e29-018b-43e5-bfa1-445d3d7e8bd4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.251772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 471/937] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:51.257056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e93e29-018b-43e5-bfa1-445d3d7e8bd4 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.321452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16f60e-1fed-4955-81ab-684dfd071e86 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.358045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beca6c8-03a2-49ff-b994-ba92450ab157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.358858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beca6c8-03a2-49ff-b994-ba92450ab157 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.359170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beca6c8-03a2-49ff-b994-ba92450ab157 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.359409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beca6c8-03a2-49ff-b994-ba92450ab157 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.360173 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beca6c8-03a2-49ff-b994-ba92450ab157 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.361683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beca6c8-03a2-49ff-b994-ba92450ab157 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.367204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e93e29-018b-43e5-bfa1-445d3d7e8bd4 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-1beca6c8-03a2-49ff-b994-ba92450ab157 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:51.375038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:51.376009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:51.376815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 467/938] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:51.382395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beca6c8-03a2-49ff-b994-ba92450ab157 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.405348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca3e43-fad1-443b-ae46-030790d12424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.406290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca3e43-fad1-443b-ae46-030790d12424 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.406599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca3e43-fad1-443b-ae46-030790d12424 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.406882 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca3e43-fad1-443b-ae46-030790d12424 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.415377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca3e43-fad1-443b-ae46-030790d12424 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.417050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca3e43-fad1-443b-ae46-030790d12424 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-e1ca3e43-fad1-443b-ae46-030790d12424 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:51.433563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:51.435733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:51.437963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:51.445906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 472/939] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:51.445906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca3e43-fad1-443b-ae46-030790d12424 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.484737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:51.492316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beca6c8-03a2-49ff-b994-ba92450ab157 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.500610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:51.528727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2daee-209b-4afd-9fbf-09e0d1572aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.529394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2daee-209b-4afd-9fbf-09e0d1572aec None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.529675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2daee-209b-4afd-9fbf-09e0d1572aec None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.529979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2daee-209b-4afd-9fbf-09e0d1572aec None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.530809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2daee-209b-4afd-9fbf-09e0d1572aec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:51.531160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2daee-209b-4afd-9fbf-09e0d1572aec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.532542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2daee-209b-4afd-9fbf-09e0d1572aec None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.542134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 468/940] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:51.544850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2daee-209b-4afd-9fbf-09e0d1572aec None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.553085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca3e43-fad1-443b-ae46-030790d12424 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.589418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614515b-9cf9-41c8-aadd-480bee2ac736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.590122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614515b-9cf9-41c8-aadd-480bee2ac736 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.590404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614515b-9cf9-41c8-aadd-480bee2ac736 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.590717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614515b-9cf9-41c8-aadd-480bee2ac736 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.591520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614515b-9cf9-41c8-aadd-480bee2ac736 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:51.591833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614515b-9cf9-41c8-aadd-480bee2ac736 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.593389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614515b-9cf9-41c8-aadd-480bee2ac736 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.612586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 473/941] 10.222.0.42 () {56 vars in 1149 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:51.618483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9690b8-b7d9-43a7-b7e1-0ba4c1fc4598 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.618776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9690b8-b7d9-43a7-b7e1-0ba4c1fc4598 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.619056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9690b8-b7d9-43a7-b7e1-0ba4c1fc4598 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.624488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 474/942] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:51.624488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9690b8-b7d9-43a7-b7e1-0ba4c1fc4598 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.650742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2daee-209b-4afd-9fbf-09e0d1572aec None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.687626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188d247-eb52-4c94-ba0d-6648d54510d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.688216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d247-eb52-4c94-ba0d-6648d54510d1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.688507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d247-eb52-4c94-ba0d-6648d54510d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.688784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d247-eb52-4c94-ba0d-6648d54510d1 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.689481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188d247-eb52-4c94-ba0d-6648d54510d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:51.689786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188d247-eb52-4c94-ba0d-6648d54510d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.691224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188d247-eb52-4c94-ba0d-6648d54510d1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.696236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 469/943] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:51.699124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f41b8-fc28-43fb-a2e8-89613420a433 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.699415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f41b8-fc28-43fb-a2e8-89613420a433 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.699693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f41b8-fc28-43fb-a2e8-89613420a433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.718628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3c7f41b8-fc28-43fb-a2e8-89613420a433 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:51.719807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3c7f41b8-fc28-43fb-a2e8-89613420a433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:51.738089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:51.739756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9690b8-b7d9-43a7-b7e1-0ba4c1fc4598 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.752873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:51.776337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8ebd6-f988-47ef-af87-efa3cdd6c4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.777135 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8ebd6-f988-47ef-af87-efa3cdd6c4fd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.777422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8ebd6-f988-47ef-af87-efa3cdd6c4fd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.777702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8ebd6-f988-47ef-af87-efa3cdd6c4fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.787557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8ebd6-f988-47ef-af87-efa3cdd6c4fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.789268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8ebd6-f988-47ef-af87-efa3cdd6c4fd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.789660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 470/944] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:51.790726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f41b8-fc28-43fb-a2e8-89613420a433 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-cfa8ebd6-f988-47ef-af87-efa3cdd6c4fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:51.816354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:51.817361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:51.818401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:51.818401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:51.818401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:51.818401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:51.818401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 475/945] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:51.824625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8ebd6-f988-47ef-af87-efa3cdd6c4fd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.904023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f41b8-fc28-43fb-a2e8-89613420a433 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.925663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8ebd6-f988-47ef-af87-efa3cdd6c4fd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:51.940537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29d4e3-4859-481b-b1b4-eae121b0b533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.941382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d4e3-4859-481b-b1b4-eae121b0b533 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.941804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d4e3-4859-481b-b1b4-eae121b0b533 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.942332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d4e3-4859-481b-b1b4-eae121b0b533 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.943251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29d4e3-4859-481b-b1b4-eae121b0b533 None admin] RBAC: Adding query filter params (interface=public, service_id=df01cbf66cf347ddbe0c18729212e624, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:51.943724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29d4e3-4859-481b-b1b4-eae121b0b533 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.945216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29d4e3-4859-481b-b1b4-eae121b0b533 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.950574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 471/946] 10.222.0.42 () {56 vars in 1251 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/endpoints?service_id=df01cbf66cf347ddbe0c18729212e624&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:51.951317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29d4e3-4859-481b-b1b4-eae121b0b533 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:51.963404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e5e28-6e63-4757-826e-546ab168863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:51.964075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5e28-6e63-4757-826e-546ab168863f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:51.964361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5e28-6e63-4757-826e-546ab168863f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:51.964643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5e28-6e63-4757-826e-546ab168863f None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:51.965374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e5e28-6e63-4757-826e-546ab168863f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:51.965683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e5e28-6e63-4757-826e-546ab168863f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:51.967084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e5e28-6e63-4757-826e-546ab168863f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:51.975266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:51.981067 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 476/947] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:51.981754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e5e28-6e63-4757-826e-546ab168863f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:52.007834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:52.062624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29d4e3-4859-481b-b1b4-eae121b0b533 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:52.086553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e5e28-6e63-4757-826e-546ab168863f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:52.101810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:52.102495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:52.102776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:52.103048 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:52.103738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:52.105159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:52.106529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WARNING keystone.server.flask.common [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 03 09:38:52.121640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb9cac-342c-4169-9811-70d59cab56b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:52.122347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb9cac-342c-4169-9811-70d59cab56b5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:52.122627 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb9cac-342c-4169-9811-70d59cab56b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:52.122903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb9cac-342c-4169-9811-70d59cab56b5 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:52.123586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb9cac-342c-4169-9811-70d59cab56b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:52.124935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb9cac-342c-4169-9811-70d59cab56b5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:52.131338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 477/948] 10.222.0.42 () {56 vars in 1087 bytes} [Fri Jun 3 09:38:51 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:38:52.132113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb9cac-342c-4169-9811-70d59cab56b5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:52.160629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.sql.core [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 03 09:38:52.160629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 03 09:38:52.160629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [parameters: {'id': '2f651c6bcbda466785de4606e1ab711e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 03 09:38:52.160629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87025) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2f651c6bcbda466785de4606e1ab711e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2f651c6bcbda466785de4606e1ab711e. Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application conn.query(q) Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 09:38:52.191615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result.read() Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 03 09:38:52.193044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application next(self.gen) Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application yield resource Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application next(self.gen) Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.session.rollback() Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.force_reraise() Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise self.value Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application session.commit() Jun 03 09:38:52.194524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._prepare_impl() Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.session.flush() Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._flush(objects) Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application compat.raise_( Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application flush_context.execute() Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 03 09:38:52.195627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rec.execute(self) Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 03 09:38:52.196516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application conn.query(q) Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result.read() Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 09:38:52.197302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [parameters: {'id': '2f651c6bcbda466785de4606e1ab711e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:52.199391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:52.201511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2f651c6bcbda466785de4606e1ab711e. Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 472/949] 10.222.0.42 () {60 vars in 1140 bytes} [Fri Jun 3 09:38:51 2022] POST /identity/v3/projects => generated 262 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:52.203711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:52.253150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:52.271574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb9cac-342c-4169-9811-70d59cab56b5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:52.309310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:52.318069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2a6a7-ab32-480f-a651-8ab4ae7e7f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:52.319470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2a6a7-ab32-480f-a651-8ab4ae7e7f3a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:52.324984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2a6a7-ab32-480f-a651-8ab4ae7e7f3a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:52.324984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2a6a7-ab32-480f-a651-8ab4ae7e7f3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:52.328307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2a6a7-ab32-480f-a651-8ab4ae7e7f3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:52.330607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2a6a7-ab32-480f-a651-8ab4ae7e7f3a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:52.340141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81199509-6c69-43c4-817a-76da1ee3961e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-1fd2a6a7-ab32-480f-a651-8ab4ae7e7f3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:52.345507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:52.346645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:52.347852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:52.347852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:52.347852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:52.347852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:52.347852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:52.347852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:52.347852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:52.347852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:52.350025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 478/950] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:52.355947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2a6a7-ab32-480f-a651-8ab4ae7e7f3a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:52.385997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca01427-c6a9-4723-8a10-bff03670d870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:52.387136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca01427-c6a9-4723-8a10-bff03670d870 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:52.387422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca01427-c6a9-4723-8a10-bff03670d870 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:52.387698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca01427-c6a9-4723-8a10-bff03670d870 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:52.397300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca01427-c6a9-4723-8a10-bff03670d870 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:52.399484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca01427-c6a9-4723-8a10-bff03670d870 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-cca01427-c6a9-4723-8a10-bff03670d870 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:52.424628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:52.425763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:52.427562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:52.427562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:52.427562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:52.427562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:52.427562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:52.427562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:52.427562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:52.427562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:52.429321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 473/951] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:52.434890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca01427-c6a9-4723-8a10-bff03670d870 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:52.475535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2a6a7-ab32-480f-a651-8ab4ae7e7f3a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:52.512417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795d9ac-b61a-4e5b-adaf-0cf912a25aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:52.513415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795d9ac-b61a-4e5b-adaf-0cf912a25aa7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:52.513953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795d9ac-b61a-4e5b-adaf-0cf912a25aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:52.514337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795d9ac-b61a-4e5b-adaf-0cf912a25aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:52.515280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795d9ac-b61a-4e5b-adaf-0cf912a25aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:52.515689 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795d9ac-b61a-4e5b-adaf-0cf912a25aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:52.517281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795d9ac-b61a-4e5b-adaf-0cf912a25aa7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:52.536075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 479/952] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:52.537290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795d9ac-b61a-4e5b-adaf-0cf912a25aa7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:52.540901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca01427-c6a9-4723-8a10-bff03670d870 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:52.546485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:52.584620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c0db7-6fb1-49dc-9db2-03b7008f44b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:52.586702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c0db7-6fb1-49dc-9db2-03b7008f44b3 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:52.587149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c0db7-6fb1-49dc-9db2-03b7008f44b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:52.587988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c0db7-6fb1-49dc-9db2-03b7008f44b3 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:52.589197 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c0db7-6fb1-49dc-9db2-03b7008f44b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:52.589658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c0db7-6fb1-49dc-9db2-03b7008f44b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:52.592116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c0db7-6fb1-49dc-9db2-03b7008f44b3 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:52.594643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:52.626097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 474/953] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:52.664074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795d9ac-b61a-4e5b-adaf-0cf912a25aa7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:52.701861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c077cc2-93f9-412d-b9dc-921c040a0f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:52.702809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c077cc2-93f9-412d-b9dc-921c040a0f6b None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:52.703089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c077cc2-93f9-412d-b9dc-921c040a0f6b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:52.703365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c077cc2-93f9-412d-b9dc-921c040a0f6b None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/fd85838dd97c4d74995302afe060388c/roles/660489b44e624266a7f8293c906652ed` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:52.705873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c077cc2-93f9-412d-b9dc-921c040a0f6b None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=fd85838dd97c4d74995302afe060388c, role_id=660489b44e624266a7f8293c906652ed)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:52.707375 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c077cc2-93f9-412d-b9dc-921c040a0f6b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:52.787183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 480/954] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:38:52 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/fd85838dd97c4d74995302afe060388c/roles/660489b44e624266a7f8293c906652ed => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:38:53.090677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd047bc7-1e9e-4081-b36e-08e1a1954d3b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:53.091235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd047bc7-1e9e-4081-b36e-08e1a1954d3b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:53.091235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd047bc7-1e9e-4081-b36e-08e1a1954d3b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:53.092830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 475/955] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:53.098287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f053e-979f-489f-a31f-7d46ad347519 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:53.098718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f053e-979f-489f-a31f-7d46ad347519 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:53.098895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f053e-979f-489f-a31f-7d46ad347519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:53.115596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0e1f053e-979f-489f-a31f-7d46ad347519 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:53.116786 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0e1f053e-979f-489f-a31f-7d46ad347519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:53.176812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:53.184507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 481/956] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:53.209044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd047bc7-1e9e-4081-b36e-08e1a1954d3b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:53.321634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd047bc7-1e9e-4081-b36e-08e1a1954d3b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:53.358334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceeba09-585a-425c-b64d-6e205043dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:53.358978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceeba09-585a-425c-b64d-6e205043dcbe None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:53.359267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceeba09-585a-425c-b64d-6e205043dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:53.359545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceeba09-585a-425c-b64d-6e205043dcbe None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:53.360281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceeba09-585a-425c-b64d-6e205043dcbe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:53.361664 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceeba09-585a-425c-b64d-6e205043dcbe None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:53.431639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 476/957] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:38:53 2022] POST /identity/v3/services => generated 242 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:38:54.484532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7c40b-a17e-4ece-9311-f34976f17e24 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:54.485081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7c40b-a17e-4ece-9311-f34976f17e24 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:54.485081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7c40b-a17e-4ece-9311-f34976f17e24 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:54.486813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 482/958] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:54.492041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddb5d0-1ee6-4011-8f5c-72aeac540116 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:54.492848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddb5d0-1ee6-4011-8f5c-72aeac540116 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:54.493354 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddb5d0-1ee6-4011-8f5c-72aeac540116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:54.511558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b9ddb5d0-1ee6-4011-8f5c-72aeac540116 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:54.513382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b9ddb5d0-1ee6-4011-8f5c-72aeac540116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:54.565534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:54.591967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 477/959] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:38:54.615253 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7c40b-a17e-4ece-9311-f34976f17e24 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:54.725905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7c40b-a17e-4ece-9311-f34976f17e24 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:54.762369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47d040-4e75-4a30-a830-7a27366ed3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:54.763095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47d040-4e75-4a30-a830-7a27366ed3e5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:54.763381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47d040-4e75-4a30-a830-7a27366ed3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:54.763657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47d040-4e75-4a30-a830-7a27366ed3e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:54.764315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47d040-4e75-4a30-a830-7a27366ed3e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:54.765686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47d040-4e75-4a30-a830-7a27366ed3e5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-fb47d040-4e75-4a30-a830-7a27366ed3e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:54.778221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:54.779291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:54.780142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 483/960] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:54.785387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddb5d0-1ee6-4011-8f5c-72aeac540116 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:54.896977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddb5d0-1ee6-4011-8f5c-72aeac540116 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:54.933034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f720270-a6da-4fd2-8a91-2f202b8be62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:54.933701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f720270-a6da-4fd2-8a91-2f202b8be62c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:54.934009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f720270-a6da-4fd2-8a91-2f202b8be62c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:54.934289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f720270-a6da-4fd2-8a91-2f202b8be62c None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:54.934998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f720270-a6da-4fd2-8a91-2f202b8be62c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:54.935306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f720270-a6da-4fd2-8a91-2f202b8be62c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:54.936689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f720270-a6da-4fd2-8a91-2f202b8be62c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:54.942050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 478/961] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:38:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:54.947861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47d040-4e75-4a30-a830-7a27366ed3e5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:55.010295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:55.057428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47d040-4e75-4a30-a830-7a27366ed3e5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:55.061555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b25e00-7a16-4e4b-be8c-1af7e2199fb0 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.061845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b25e00-7a16-4e4b-be8c-1af7e2199fb0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.062153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b25e00-7a16-4e4b-be8c-1af7e2199fb0 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.063739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 479/962] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:55.069099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f5702-c13a-4d6c-a705-9ba3b5c20fe8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.069667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f5702-c13a-4d6c-a705-9ba3b5c20fe8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.070198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f5702-c13a-4d6c-a705-9ba3b5c20fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.087696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b35f5702-c13a-4d6c-a705-9ba3b5c20fe8 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:55.089193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b35f5702-c13a-4d6c-a705-9ba3b5c20fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:55.094402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce409fb6-9e3b-4c17-bc49-604589f3ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:55.095071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce409fb6-9e3b-4c17-bc49-604589f3ddf9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.095358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce409fb6-9e3b-4c17-bc49-604589f3ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.095637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce409fb6-9e3b-4c17-bc49-604589f3ddf9 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.096348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce409fb6-9e3b-4c17-bc49-604589f3ddf9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:55.096657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce409fb6-9e3b-4c17-bc49-604589f3ddf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:55.098078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce409fb6-9e3b-4c17-bc49-604589f3ddf9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:55.104590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 484/963] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:38:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:55.109783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce409fb6-9e3b-4c17-bc49-604589f3ddf9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:55.135036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:55.169975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 480/964] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:38:55.189001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f5702-c13a-4d6c-a705-9ba3b5c20fe8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:55.246439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce409fb6-9e3b-4c17-bc49-604589f3ddf9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:55.272523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:55.286243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf21a6-efec-4365-9890-c446a9041a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:55.287068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf21a6-efec-4365-9890-c446a9041a04 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.287351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf21a6-efec-4365-9890-c446a9041a04 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.287951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf21a6-efec-4365-9890-c446a9041a04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.289079 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf21a6-efec-4365-9890-c446a9041a04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:55.291048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf21a6-efec-4365-9890-c446a9041a04 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:55.313380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f5702-c13a-4d6c-a705-9ba3b5c20fe8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:55.349168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:55.349828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.350138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.350412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.351104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:55.352539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:55.390345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 485/965] 10.222.0.42 () {60 vars in 1144 bytes} [Fri Jun 3 09:38:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:38:55.390345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a429d2-be71-46a4-811e-53051f983c73 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.390345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a429d2-be71-46a4-811e-53051f983c73 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.390345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a429d2-be71-46a4-811e-53051f983c73 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.390345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 486/966] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:55.394532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba957e2-aa2b-4900-8601-3991d1a0fc78 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.394816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba957e2-aa2b-4900-8601-3991d1a0fc78 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.395138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba957e2-aa2b-4900-8601-3991d1a0fc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.414859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6ba957e2-aa2b-4900-8601-3991d1a0fc78 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:55.416078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6ba957e2-aa2b-4900-8601-3991d1a0fc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:55.442174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.sql.core [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 03 09:38:55.442174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 03 09:38:55.442174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [parameters: {'user_id': 'fbf4831d14474fc49b72cb73a19de340', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 03 09:38:55.442174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87025) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application conn.query(q) Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result.read() Jun 03 09:38:55.450094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 09:38:55.451070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application next(self.gen) Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application yield resource Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application next(self.gen) Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.session.rollback() Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.force_reraise() Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise self.value Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application session.commit() Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 03 09:38:55.452047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._prepare_impl() Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.session.flush() Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._flush(objects) Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application compat.raise_( Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application flush_context.execute() Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rec.execute(self) Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 03 09:38:55.453026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 09:38:55.454666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application conn.query(q) Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application result.read() Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 09:38:55.457004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fbf4831d14474fc49b72cb73a19de340', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:55.459637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 03 09:38:55.460551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 03 09:38:55.461523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 03 09:38:55.461523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 03 09:38:55.461523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 03 09:38:55.461523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 03 09:38:55.461523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 03 09:38:55.461523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:55.461523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 481/967] 10.222.0.42 () {60 vars in 1131 bytes} [Fri Jun 3 09:38:55 2022] POST /identity/v3/users => generated 160 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:55.461523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:55.522371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 487/968] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:38:55.526413 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:55.544123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba957e2-aa2b-4900-8601-3991d1a0fc78 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:55.600273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d03ca-cd19-4635-b273-4794dde9b2da None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:55.647441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b633662-8347-453d-b0cd-59fac8e09302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:55.648513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b633662-8347-453d-b0cd-59fac8e09302 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.648954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b633662-8347-453d-b0cd-59fac8e09302 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.649391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b633662-8347-453d-b0cd-59fac8e09302 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.659532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b633662-8347-453d-b0cd-59fac8e09302 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:55.664425 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba957e2-aa2b-4900-8601-3991d1a0fc78 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:55.664556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b633662-8347-453d-b0cd-59fac8e09302 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:55.671064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-2b633662-8347-453d-b0cd-59fac8e09302 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:55.678628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:55.679579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:55.680585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 482/969] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:38:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:55.685797 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b633662-8347-453d-b0cd-59fac8e09302 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:55.717397 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0203ad-8e21-4c5f-aaee-722072366584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:55.717658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0203ad-8e21-4c5f-aaee-722072366584 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.717973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0203ad-8e21-4c5f-aaee-722072366584 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.718257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0203ad-8e21-4c5f-aaee-722072366584 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.733365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0203ad-8e21-4c5f-aaee-722072366584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-3f0203ad-8e21-4c5f-aaee-722072366584 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:38:55.745312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:55.746429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:38:55.747269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:55.747929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 488/970] 10.222.0.42 () {56 vars in 1120 bytes} [Fri Jun 3 09:38:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:55.754425 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0203ad-8e21-4c5f-aaee-722072366584 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:55.806978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:55.814338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b633662-8347-453d-b0cd-59fac8e09302 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:55.851910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a3b40-0ef0-483b-8f69-e939ef48d288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:55.852506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a3b40-0ef0-483b-8f69-e939ef48d288 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.852838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a3b40-0ef0-483b-8f69-e939ef48d288 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.853124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a3b40-0ef0-483b-8f69-e939ef48d288 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.853966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a3b40-0ef0-483b-8f69-e939ef48d288 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:55.854277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a3b40-0ef0-483b-8f69-e939ef48d288 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:55.855820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a3b40-0ef0-483b-8f69-e939ef48d288 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:55.876224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0203ad-8e21-4c5f-aaee-722072366584 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:55.894584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 483/971] 10.222.0.42 () {56 vars in 1101 bytes} [Fri Jun 3 09:38:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:55.896560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca051a-4c50-46d5-a961-51b1e4020dd2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.896844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca051a-4c50-46d5-a961-51b1e4020dd2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.897119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca051a-4c50-46d5-a961-51b1e4020dd2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.898585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 484/972] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:55.904170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9666ba-e9ef-47dd-aea4-8d3dce4fa1ff None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.904474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9666ba-e9ef-47dd-aea4-8d3dce4fa1ff None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.904749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9666ba-e9ef-47dd-aea4-8d3dce4fa1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.921982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c923c6-60fd-434d-b7f6-9d668468da1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:55.923077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c923c6-60fd-434d-b7f6-9d668468da1f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:55.923560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c923c6-60fd-434d-b7f6-9d668468da1f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:55.924065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c923c6-60fd-434d-b7f6-9d668468da1f None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:55.924182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fd9666ba-e9ef-47dd-aea4-8d3dce4fa1ff None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:55.925148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c923c6-60fd-434d-b7f6-9d668468da1f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:55.925689 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c923c6-60fd-434d-b7f6-9d668468da1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:55.925803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fd9666ba-e9ef-47dd-aea4-8d3dce4fa1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:55.927659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c923c6-60fd-434d-b7f6-9d668468da1f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:55.944744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 489/973] 10.222.0.42 () {56 vars in 1115 bytes} [Fri Jun 3 09:38:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:55.950541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c923c6-60fd-434d-b7f6-9d668468da1f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:55.983212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:56.032783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 485/974] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:38:56.060581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9666ba-e9ef-47dd-aea4-8d3dce4fa1ff None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:56.091161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c923c6-60fd-434d-b7f6-9d668468da1f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.141492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dd7e1-0cb4-498b-a37a-4e8a3df7b5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.142263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dd7e1-0cb4-498b-a37a-4e8a3df7b5c5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.142551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dd7e1-0cb4-498b-a37a-4e8a3df7b5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.142827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9dd7e1-0cb4-498b-a37a-4e8a3df7b5c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.153279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9dd7e1-0cb4-498b-a37a-4e8a3df7b5c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.154881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9dd7e1-0cb4-498b-a37a-4e8a3df7b5c5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:56.162629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-be9dd7e1-0cb4-498b-a37a-4e8a3df7b5c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:56.175457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:56.178671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:56.179681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:56.179681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:56.179681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:56.179681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:56.179681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 490/975] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:56.182876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dd7e1-0cb4-498b-a37a-4e8a3df7b5c5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:56.187136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9666ba-e9ef-47dd-aea4-8d3dce4fa1ff None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.232571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8cdae-5213-4a03-b9dc-1a6b88184cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.233369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8cdae-5213-4a03-b9dc-1a6b88184cfc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.233647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8cdae-5213-4a03-b9dc-1a6b88184cfc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.233951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8cdae-5213-4a03-b9dc-1a6b88184cfc None admin] PATH_INFO: `/v3/services/compute` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.234605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8cdae-5213-4a03-b9dc-1a6b88184cfc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.235999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8cdae-5213-4a03-b9dc-1a6b88184cfc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0af8cdae-5213-4a03-b9dc-1a6b88184cfc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:56.250458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:56.251627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:56.252487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 486/976] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:56.260112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8cdae-5213-4a03-b9dc-1a6b88184cfc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:56.294294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:56.313623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9dd7e1-0cb4-498b-a37a-4e8a3df7b5c5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.352084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3726051-4a38-41bf-9728-d3ae94657931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.352893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3726051-4a38-41bf-9728-d3ae94657931 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.353174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3726051-4a38-41bf-9728-d3ae94657931 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.353449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3726051-4a38-41bf-9728-d3ae94657931 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.354196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3726051-4a38-41bf-9728-d3ae94657931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:56.354494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3726051-4a38-41bf-9728-d3ae94657931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.355861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3726051-4a38-41bf-9728-d3ae94657931 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:56.363346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8cdae-5213-4a03-b9dc-1a6b88184cfc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.367872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 491/977] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:56.373107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3726051-4a38-41bf-9728-d3ae94657931 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:56.398635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b68af0-bd5c-459b-bc50-ee3ec71b8307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.399325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b68af0-bd5c-459b-bc50-ee3ec71b8307 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.399607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b68af0-bd5c-459b-bc50-ee3ec71b8307 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.399939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b68af0-bd5c-459b-bc50-ee3ec71b8307 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.400640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b68af0-bd5c-459b-bc50-ee3ec71b8307 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:56.400960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b68af0-bd5c-459b-bc50-ee3ec71b8307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.402367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b68af0-bd5c-459b-bc50-ee3ec71b8307 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:56.409922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 487/978] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:56.412759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b68af0-bd5c-459b-bc50-ee3ec71b8307 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:56.429089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:56.482620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3726051-4a38-41bf-9728-d3ae94657931 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.518104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b68af0-bd5c-459b-bc50-ee3ec71b8307 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.518996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16792612-13b5-44b3-83bc-d33dcc282a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.519798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16792612-13b5-44b3-83bc-d33dcc282a45 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.520615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16792612-13b5-44b3-83bc-d33dcc282a45 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.521560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16792612-13b5-44b3-83bc-d33dcc282a45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.528693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16792612-13b5-44b3-83bc-d33dcc282a45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.530158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16792612-13b5-44b3-83bc-d33dcc282a45 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:56.533491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-16792612-13b5-44b3-83bc-d33dcc282a45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:38:56.546594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:56.547576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:56.548502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 492/979] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:38:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:38:56.553295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16792612-13b5-44b3-83bc-d33dcc282a45 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:56.557920 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8c451-ce6b-40ee-91d6-43e846d2397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.558583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8c451-ce6b-40ee-91d6-43e846d2397a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.558866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8c451-ce6b-40ee-91d6-43e846d2397a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.559145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8c451-ce6b-40ee-91d6-43e846d2397a None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.559847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8c451-ce6b-40ee-91d6-43e846d2397a None admin] RBAC: Adding query filter params (type=compute) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:56.560153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8c451-ce6b-40ee-91d6-43e846d2397a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.561587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8c451-ce6b-40ee-91d6-43e846d2397a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:56.566785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 488/980] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:56.572215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8c451-ce6b-40ee-91d6-43e846d2397a None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:56.653199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:56.663088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16792612-13b5-44b3-83bc-d33dcc282a45 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.677134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8c451-ce6b-40ee-91d6-43e846d2397a None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.700418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36956847-e331-4970-a7c6-1ec5bc243d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.701257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36956847-e331-4970-a7c6-1ec5bc243d29 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.701550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36956847-e331-4970-a7c6-1ec5bc243d29 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.702051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36956847-e331-4970-a7c6-1ec5bc243d29 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.702943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36956847-e331-4970-a7c6-1ec5bc243d29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:56.703303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36956847-e331-4970-a7c6-1ec5bc243d29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.704747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36956847-e331-4970-a7c6-1ec5bc243d29 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:56.720864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7d7d3-90b8-412f-a015-dc83f3ef09d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.721810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d7d3-90b8-412f-a015-dc83f3ef09d6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.722138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d7d3-90b8-412f-a015-dc83f3ef09d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.722418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d7d3-90b8-412f-a015-dc83f3ef09d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.723698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7d7d3-90b8-412f-a015-dc83f3ef09d6 None admin] RBAC: Adding query filter params (interface=public, service_id=9e64dd017d684249a7e0772c7e24f7ee, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:56.724007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7d7d3-90b8-412f-a015-dc83f3ef09d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.724511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 493/981] 10.222.0.42 () {56 vars in 1149 bytes} [Fri Jun 3 09:38:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:56.727219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7d7d3-90b8-412f-a015-dc83f3ef09d6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:56.729629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36956847-e331-4970-a7c6-1ec5bc243d29 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:56.737609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 489/982] 10.222.0.42 () {56 vars in 1251 bytes} [Fri Jun 3 09:38:56 2022] GET /identity/v3/endpoints?service_id=9e64dd017d684249a7e0772c7e24f7ee&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:56.738509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7d7d3-90b8-412f-a015-dc83f3ef09d6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:56.792323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:56.836621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36956847-e331-4970-a7c6-1ec5bc243d29 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.842361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7d7d3-90b8-412f-a015-dc83f3ef09d6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:56.872906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da5c6d-c0e4-4082-9052-fd01721490c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.873549 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da5c6d-c0e4-4082-9052-fd01721490c8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.873828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da5c6d-c0e4-4082-9052-fd01721490c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.874139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da5c6d-c0e4-4082-9052-fd01721490c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.876316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac3b0d-28d6-4ef3-a4c4-1792d54e84bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:56.876978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3b0d-28d6-4ef3-a4c4-1792d54e84bd None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:56.877268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3b0d-28d6-4ef3-a4c4-1792d54e84bd None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:56.877545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3b0d-28d6-4ef3-a4c4-1792d54e84bd None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:56.878255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac3b0d-28d6-4ef3-a4c4-1792d54e84bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.880189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac3b0d-28d6-4ef3-a4c4-1792d54e84bd None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:56.886557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 490/983] 10.222.0.42 () {56 vars in 1087 bytes} [Fri Jun 3 09:38:56 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:38:56.961539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da5c6d-c0e4-4082-9052-fd01721490c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:56.962958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da5c6d-c0e4-4082-9052-fd01721490c8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:57.023116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:57.069184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 494/984] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:38:56 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 09:38:57.075359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac3b0d-28d6-4ef3-a4c4-1792d54e84bd None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:57.079326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:57.177004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac3b0d-28d6-4ef3-a4c4-1792d54e84bd None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:57.210141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f0411-9846-47fa-bd3d-b04e4f5e3a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:57.210923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f0411-9846-47fa-bd3d-b04e4f5e3a98 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:57.211202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f0411-9846-47fa-bd3d-b04e4f5e3a98 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:57.211469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f0411-9846-47fa-bd3d-b04e4f5e3a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:57.222038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f0411-9846-47fa-bd3d-b04e4f5e3a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:57.223457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f0411-9846-47fa-bd3d-b04e4f5e3a98 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-445f0411-9846-47fa-bd3d-b04e4f5e3a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:38:57.240742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:57.241649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:57.247517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:38:57.247517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:38:57.247517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:38:57.247517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:57.247517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 491/985] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:57.247816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da5c6d-c0e4-4082-9052-fd01721490c8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:57.346269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da5c6d-c0e4-4082-9052-fd01721490c8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:57.380687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bfd3c-d9e3-4f33-824e-fdd1c7bdaf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:57.381347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bfd3c-d9e3-4f33-824e-fdd1c7bdaf53 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:57.381623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bfd3c-d9e3-4f33-824e-fdd1c7bdaf53 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:57.381913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bfd3c-d9e3-4f33-824e-fdd1c7bdaf53 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:57.382627 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bfd3c-d9e3-4f33-824e-fdd1c7bdaf53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:57.382926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bfd3c-d9e3-4f33-824e-fdd1c7bdaf53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:57.384280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bfd3c-d9e3-4f33-824e-fdd1c7bdaf53 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:57.396508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 495/986] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:57.401439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f0411-9846-47fa-bd3d-b04e4f5e3a98 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:57.461619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:57.505685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f0411-9846-47fa-bd3d-b04e4f5e3a98 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:57.538923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae59a5a-121a-42e4-ad31-16387fdd8f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:57.539711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae59a5a-121a-42e4-ad31-16387fdd8f16 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:57.539989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae59a5a-121a-42e4-ad31-16387fdd8f16 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:57.540259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae59a5a-121a-42e4-ad31-16387fdd8f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:57.546801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae59a5a-121a-42e4-ad31-16387fdd8f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:57.548230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae59a5a-121a-42e4-ad31-16387fdd8f16 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-7ae59a5a-121a-42e4-ad31-16387fdd8f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:38:57.562313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:57.563627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:57.564511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:38:57.564511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:38:57.564511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:38:57.564511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:38:57.564511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:38:57.564511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:38:57.564511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:38:57.564511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:57.566205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 492/987] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:57.569661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bfd3c-d9e3-4f33-824e-fdd1c7bdaf53 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:57.589756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:57.669043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bfd3c-d9e3-4f33-824e-fdd1c7bdaf53 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:57.702114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a9f80-a74f-4512-a8cc-d5d26b9b0287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:57.702769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9f80-a74f-4512-a8cc-d5d26b9b0287 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:57.703047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9f80-a74f-4512-a8cc-d5d26b9b0287 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:57.703320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a9f80-a74f-4512-a8cc-d5d26b9b0287 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:57.704098 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a9f80-a74f-4512-a8cc-d5d26b9b0287 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:57.704403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a9f80-a74f-4512-a8cc-d5d26b9b0287 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:57.705822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a9f80-a74f-4512-a8cc-d5d26b9b0287 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:57.721075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 496/988] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:38:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:57.727379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae59a5a-121a-42e4-ad31-16387fdd8f16 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:57.774147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb5b03-98bd-41ad-8e76-4adb8f6afe01 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:57.774423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb5b03-98bd-41ad-8e76-4adb8f6afe01 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:57.774636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb5b03-98bd-41ad-8e76-4adb8f6afe01 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:57.776110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 497/989] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:38:57.780885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f886e9a-ad07-49f0-9ce2-4e0c0d2334e3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:57.781172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f886e9a-ad07-49f0-9ce2-4e0c0d2334e3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:57.781450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f886e9a-ad07-49f0-9ce2-4e0c0d2334e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:57.797818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6f886e9a-ad07-49f0-9ce2-4e0c0d2334e3 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:57.799037 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6f886e9a-ad07-49f0-9ce2-4e0c0d2334e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:57.832562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae59a5a-121a-42e4-ad31-16387fdd8f16 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:57.849276 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:57.867501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef18a-dbb6-4856-be3f-e37fade1e723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:57.868160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef18a-dbb6-4856-be3f-e37fade1e723 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:57.868447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef18a-dbb6-4856-be3f-e37fade1e723 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:57.868726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ef18a-dbb6-4856-be3f-e37fade1e723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:57.869556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ef18a-dbb6-4856-be3f-e37fade1e723 None admin] RBAC: Adding query filter params (role.id=660489b44e624266a7f8293c906652ed, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=fd85838dd97c4d74995302afe060388c) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:57.869869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ef18a-dbb6-4856-be3f-e37fade1e723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:57.871325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ef18a-dbb6-4856-be3f-e37fade1e723 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:57.886149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 493/990] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:38:57 2022] GET /identity/v3/role_assignments?user.id=fd85838dd97c4d74995302afe060388c&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=660489b44e624266a7f8293c906652ed => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:38:57.886354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 498/991] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:38:57.890416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef18a-dbb6-4856-be3f-e37fade1e723 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:58.000026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ef18a-dbb6-4856-be3f-e37fade1e723 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:58.037764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64c4c6-5fe0-43ff-8573-b2cd647ec6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:58.038773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64c4c6-5fe0-43ff-8573-b2cd647ec6a4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:58.039068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64c4c6-5fe0-43ff-8573-b2cd647ec6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:58.039343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64c4c6-5fe0-43ff-8573-b2cd647ec6a4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:58.040011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64c4c6-5fe0-43ff-8573-b2cd647ec6a4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:58.041409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64c4c6-5fe0-43ff-8573-b2cd647ec6a4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3c64c4c6-5fe0-43ff-8573-b2cd647ec6a4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:58.054778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:58.055853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:38:58.056691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:38:58.061280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 494/992] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:38:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:58.062660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f886e9a-ad07-49f0-9ce2-4e0c0d2334e3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:58.076341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:58.167700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f886e9a-ad07-49f0-9ce2-4e0c0d2334e3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:58.201308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd81bfa-cb28-4ec3-90bc-d1f5d2e6455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:58.201992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd81bfa-cb28-4ec3-90bc-d1f5d2e6455e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:58.202273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd81bfa-cb28-4ec3-90bc-d1f5d2e6455e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:58.202546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd81bfa-cb28-4ec3-90bc-d1f5d2e6455e None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:58.203241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd81bfa-cb28-4ec3-90bc-d1f5d2e6455e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:58.203545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd81bfa-cb28-4ec3-90bc-d1f5d2e6455e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:58.204905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd81bfa-cb28-4ec3-90bc-d1f5d2e6455e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:58.211361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 499/993] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:38:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:58.216213 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64c4c6-5fe0-43ff-8573-b2cd647ec6a4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:58.261194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:58.315833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64c4c6-5fe0-43ff-8573-b2cd647ec6a4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:58.349705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0aca55-1eee-4c94-9be1-8abb1b86305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:58.350639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0aca55-1eee-4c94-9be1-8abb1b86305d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:58.351098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0aca55-1eee-4c94-9be1-8abb1b86305d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:58.351527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0aca55-1eee-4c94-9be1-8abb1b86305d None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:58.352359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0aca55-1eee-4c94-9be1-8abb1b86305d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:58.352806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0aca55-1eee-4c94-9be1-8abb1b86305d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:58.354340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0aca55-1eee-4c94-9be1-8abb1b86305d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:58.364760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 495/994] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:38:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:58.366299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd81bfa-cb28-4ec3-90bc-d1f5d2e6455e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:58.438068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:58.467326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd81bfa-cb28-4ec3-90bc-d1f5d2e6455e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:58.501511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1452273-2a6a-442c-a746-e782b928875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:58.502339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452273-2a6a-442c-a746-e782b928875f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:58.502716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452273-2a6a-442c-a746-e782b928875f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:58.503084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452273-2a6a-442c-a746-e782b928875f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:58.503973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452273-2a6a-442c-a746-e782b928875f None admin] RBAC: Adding query filter params (interface=admin, service_id=df01cbf66cf347ddbe0c18729212e624, region_id=RegionOne) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:58.504438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452273-2a6a-442c-a746-e782b928875f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:58.505906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452273-2a6a-442c-a746-e782b928875f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:58.511306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 500/995] 10.222.0.42 () {56 vars in 1249 bytes} [Fri Jun 3 09:38:58 2022] GET /identity/v3/endpoints?service_id=df01cbf66cf347ddbe0c18729212e624&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:58.516566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0aca55-1eee-4c94-9be1-8abb1b86305d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:58.618319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0aca55-1eee-4c94-9be1-8abb1b86305d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:58.627153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:58.651682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae644b-d5d8-477b-b9a1-87b1df46a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:58.652354 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae644b-d5d8-477b-b9a1-87b1df46a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:58.652637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae644b-d5d8-477b-b9a1-87b1df46a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:58.652912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae644b-d5d8-477b-b9a1-87b1df46a8c7 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:58.653580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae644b-d5d8-477b-b9a1-87b1df46a8c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:58.654974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae644b-d5d8-477b-b9a1-87b1df46a8c7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:58.669961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 496/996] 10.222.0.42 () {56 vars in 1087 bytes} [Fri Jun 3 09:38:58 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:38:59.175972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751739c-a8ac-4c50-a3d3-997301a2a24e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:59.176607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751739c-a8ac-4c50-a3d3-997301a2a24e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:59.176607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751739c-a8ac-4c50-a3d3-997301a2a24e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:59.178194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 501/997] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:38:59.183661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958f25f-e279-4a22-bc39-1eeb24f95bc9 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:59.183941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958f25f-e279-4a22-bc39-1eeb24f95bc9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:59.184227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958f25f-e279-4a22-bc39-1eeb24f95bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:59.201988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5958f25f-e279-4a22-bc39-1eeb24f95bc9 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:38:59.203112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5958f25f-e279-4a22-bc39-1eeb24f95bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:38:59.269801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 497/998] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:38:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:38:59.292478 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751739c-a8ac-4c50-a3d3-997301a2a24e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:59.391794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751739c-a8ac-4c50-a3d3-997301a2a24e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:59.425356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100efe85-1e16-4397-88b6-91d846152d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:59.426759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100efe85-1e16-4397-88b6-91d846152d7f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:59.426994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100efe85-1e16-4397-88b6-91d846152d7f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:59.427270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100efe85-1e16-4397-88b6-91d846152d7f None admin] PATH_INFO: `/v3/services/compute` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:59.427915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100efe85-1e16-4397-88b6-91d846152d7f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:59.429281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100efe85-1e16-4397-88b6-91d846152d7f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:59.431835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-100efe85-1e16-4397-88b6-91d846152d7f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:38:59.441911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:38:59.454448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:38:59.455314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 502/999] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:38:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:38:59.456118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958f25f-e279-4a22-bc39-1eeb24f95bc9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:59.548183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958f25f-e279-4a22-bc39-1eeb24f95bc9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:59.549705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:38:59.581864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda4e16-8cea-4c69-a1c4-f99974594aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:59.583061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda4e16-8cea-4c69-a1c4-f99974594aa0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:59.583345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda4e16-8cea-4c69-a1c4-f99974594aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:59.583618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda4e16-8cea-4c69-a1c4-f99974594aa0 None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:59.584315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda4e16-8cea-4c69-a1c4-f99974594aa0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:59.584620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda4e16-8cea-4c69-a1c4-f99974594aa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:59.586003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda4e16-8cea-4c69-a1c4-f99974594aa0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:59.591372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 498/1000] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:59.596875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100efe85-1e16-4397-88b6-91d846152d7f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:59.697857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100efe85-1e16-4397-88b6-91d846152d7f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:59.738209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1914b-594d-41e2-ab25-4abf14442b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:59.738906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1914b-594d-41e2-ab25-4abf14442b4e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:59.739186 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1914b-594d-41e2-ab25-4abf14442b4e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:59.739456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1914b-594d-41e2-ab25-4abf14442b4e None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:59.740153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1914b-594d-41e2-ab25-4abf14442b4e None admin] RBAC: Adding query filter params (type=compute) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:38:59.740454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1914b-594d-41e2-ab25-4abf14442b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:59.741804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1914b-594d-41e2-ab25-4abf14442b4e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:38:59.751162 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 503/1001] 10.222.0.42 () {56 vars in 1112 bytes} [Fri Jun 3 09:38:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:38:59.752362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda4e16-8cea-4c69-a1c4-f99974594aa0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:38:59.854377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda4e16-8cea-4c69-a1c4-f99974594aa0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:38:59.890600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df1430-fd71-4fc5-ad14-bb615c0a139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:38:59.891293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df1430-fd71-4fc5-ad14-bb615c0a139b None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:38:59.891580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df1430-fd71-4fc5-ad14-bb615c0a139b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:38:59.891861 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df1430-fd71-4fc5-ad14-bb615c0a139b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:38:59.892623 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df1430-fd71-4fc5-ad14-bb615c0a139b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:38:59.894081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df1430-fd71-4fc5-ad14-bb615c0a139b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:00.013988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 499/1002] 10.222.0.42 () {60 vars in 1144 bytes} [Fri Jun 3 09:38:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:39:00.222282 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacbf5e-891c-4ab3-ba37-e87f2e3bbe63 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:00.222794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacbf5e-891c-4ab3-ba37-e87f2e3bbe63 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:00.222889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacbf5e-891c-4ab3-ba37-e87f2e3bbe63 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:00.224749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 504/1003] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:00.229976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad958c-40a6-4cdc-83eb-4055654a922a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:00.230275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad958c-40a6-4cdc-83eb-4055654a922a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:00.230564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad958c-40a6-4cdc-83eb-4055654a922a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:00.247562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c1ad958c-40a6-4cdc-83eb-4055654a922a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:00.248761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c1ad958c-40a6-4cdc-83eb-4055654a922a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:00.265993 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:00.325463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 500/1004] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:00 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:00.347583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacbf5e-891c-4ab3-ba37-e87f2e3bbe63 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:00.413032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:00.452294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacbf5e-891c-4ab3-ba37-e87f2e3bbe63 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:00.486146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc732922-3f63-4d28-b6e2-ba3d5184eeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:00.486936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc732922-3f63-4d28-b6e2-ba3d5184eeba None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:00.487212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc732922-3f63-4d28-b6e2-ba3d5184eeba None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:00.487481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc732922-3f63-4d28-b6e2-ba3d5184eeba None admin] PATH_INFO: `/v3/services/image` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:00.488129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc732922-3f63-4d28-b6e2-ba3d5184eeba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:00.489513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc732922-3f63-4d28-b6e2-ba3d5184eeba None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-fc732922-3f63-4d28-b6e2-ba3d5184eeba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:00.501181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:00.502418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:00.503283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 505/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Fri Jun 3 09:39:00 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:00.509360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad958c-40a6-4cdc-83eb-4055654a922a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:00.609689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad958c-40a6-4cdc-83eb-4055654a922a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:00.645608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b3e48-bc05-4fdf-8b6b-bec2f329ebba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:00.646330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b3e48-bc05-4fdf-8b6b-bec2f329ebba None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:00.646613 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b3e48-bc05-4fdf-8b6b-bec2f329ebba None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:00.646895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b3e48-bc05-4fdf-8b6b-bec2f329ebba None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:00.647619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b3e48-bc05-4fdf-8b6b-bec2f329ebba None admin] RBAC: Adding query filter params (name=image) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:00.647924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b3e48-bc05-4fdf-8b6b-bec2f329ebba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:00.649349 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b3e48-bc05-4fdf-8b6b-bec2f329ebba None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:00.659923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 501/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:00.661532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc732922-3f63-4d28-b6e2-ba3d5184eeba None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:00.768934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc732922-3f63-4d28-b6e2-ba3d5184eeba None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:00.795452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:00.803713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde65a30-e26f-4587-b234-2b15b0e724f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:00.804376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde65a30-e26f-4587-b234-2b15b0e724f4 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:00.804653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde65a30-e26f-4587-b234-2b15b0e724f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:00.804925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde65a30-e26f-4587-b234-2b15b0e724f4 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:00.805625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde65a30-e26f-4587-b234-2b15b0e724f4 None admin] RBAC: Adding query filter params (type=image) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:00.805952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde65a30-e26f-4587-b234-2b15b0e724f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:00.807407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde65a30-e26f-4587-b234-2b15b0e724f4 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:00.816784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 506/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:39:00.987596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bffc59-2a9b-46d8-9f94-a61ce832fac7 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:00.988096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bffc59-2a9b-46d8-9f94-a61ce832fac7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:00.988096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bffc59-2a9b-46d8-9f94-a61ce832fac7 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:00.990202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 502/1008] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:00.995765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662795b-6b5f-408a-ba30-93c542fa5df2 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:00.996056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662795b-6b5f-408a-ba30-93c542fa5df2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:00.996336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662795b-6b5f-408a-ba30-93c542fa5df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:01.014061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3662795b-6b5f-408a-ba30-93c542fa5df2 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:01.015234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3662795b-6b5f-408a-ba30-93c542fa5df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:01.085709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 507/1009] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:00 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:01.108166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bffc59-2a9b-46d8-9f94-a61ce832fac7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:01.123454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:01.209129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bffc59-2a9b-46d8-9f94-a61ce832fac7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:01.242574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da555a-f2f0-42ff-bbad-35cb12eb5df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:01.243311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da555a-f2f0-42ff-bbad-35cb12eb5df1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:01.243591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da555a-f2f0-42ff-bbad-35cb12eb5df1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:01.243925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da555a-f2f0-42ff-bbad-35cb12eb5df1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:01.244584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da555a-f2f0-42ff-bbad-35cb12eb5df1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:01.246051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da555a-f2f0-42ff-bbad-35cb12eb5df1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-70da555a-f2f0-42ff-bbad-35cb12eb5df1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:01.259034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:01.259996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 09:39:01.260826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:01.265240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 503/1010] 10.222.0.42 () {56 vars in 1126 bytes} [Fri Jun 3 09:39:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:01.266264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662795b-6b5f-408a-ba30-93c542fa5df2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:01.375033 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662795b-6b5f-408a-ba30-93c542fa5df2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:01.397502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:01.409051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83aabd-14ef-4e0f-89f1-2c7a2f23134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:01.409716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83aabd-14ef-4e0f-89f1-2c7a2f23134a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:01.410044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83aabd-14ef-4e0f-89f1-2c7a2f23134a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:01.410310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83aabd-14ef-4e0f-89f1-2c7a2f23134a None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:01.411538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83aabd-14ef-4e0f-89f1-2c7a2f23134a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:01.411841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83aabd-14ef-4e0f-89f1-2c7a2f23134a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:01.413187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83aabd-14ef-4e0f-89f1-2c7a2f23134a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:01.418395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 508/1011] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:39:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:01.423767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da555a-f2f0-42ff-bbad-35cb12eb5df1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:01.500297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:01.524071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da555a-f2f0-42ff-bbad-35cb12eb5df1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:01.557384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0505a-6bee-429b-9b46-95fc2fef374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:01.558070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0505a-6bee-429b-9b46-95fc2fef374d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:01.558352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0505a-6bee-429b-9b46-95fc2fef374d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:01.558627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0505a-6bee-429b-9b46-95fc2fef374d None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:01.559374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0505a-6bee-429b-9b46-95fc2fef374d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:01.559682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0505a-6bee-429b-9b46-95fc2fef374d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:01.561041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0505a-6bee-429b-9b46-95fc2fef374d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:01.570480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 504/1012] 10.222.0.42 () {56 vars in 1122 bytes} [Fri Jun 3 09:39:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:01.571481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83aabd-14ef-4e0f-89f1-2c7a2f23134a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:01.670272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83aabd-14ef-4e0f-89f1-2c7a2f23134a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:01.702872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383b8e8-1dae-4dd1-a476-02834163391e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:01.703525 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383b8e8-1dae-4dd1-a476-02834163391e None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:01.703805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383b8e8-1dae-4dd1-a476-02834163391e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:01.704079 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383b8e8-1dae-4dd1-a476-02834163391e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:01.704805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383b8e8-1dae-4dd1-a476-02834163391e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:01.706176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383b8e8-1dae-4dd1-a476-02834163391e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:01.774887 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 509/1013] 10.222.0.42 () {60 vars in 1144 bytes} [Fri Jun 3 09:39:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:39:02.366687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94675fca-10be-4ce0-8482-1a796e843b91 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:02.367208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94675fca-10be-4ce0-8482-1a796e843b91 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:02.367290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94675fca-10be-4ce0-8482-1a796e843b91 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:02.368734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 505/1014] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:02.373791 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2fc5-ca1c-4131-bdce-37097b401c75 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:02.374212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2fc5-ca1c-4131-bdce-37097b401c75 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:02.374406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f2fc5-ca1c-4131-bdce-37097b401c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:02.391528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a20f2fc5-ca1c-4131-bdce-37097b401c75 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:02.392693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a20f2fc5-ca1c-4131-bdce-37097b401c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:02.434498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:02.464524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 510/1015] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:02 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:02.488070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94675fca-10be-4ce0-8482-1a796e843b91 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:02.590851 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94675fca-10be-4ce0-8482-1a796e843b91 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:02.624256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d1105-4a6d-4846-b953-0fcbf617aa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:02.624989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d1105-4a6d-4846-b953-0fcbf617aa49 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:02.625270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d1105-4a6d-4846-b953-0fcbf617aa49 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:02.625544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d1105-4a6d-4846-b953-0fcbf617aa49 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:02.633683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:02.637440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d1105-4a6d-4846-b953-0fcbf617aa49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-820d1105-4a6d-4846-b953-0fcbf617aa49 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:02.646485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:02.647612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:39:02.648435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:02.650202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 506/1016] 10.222.0.42 () {56 vars in 1120 bytes} [Fri Jun 3 09:39:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:02.653690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f2fc5-ca1c-4131-bdce-37097b401c75 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:02.759897 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f2fc5-ca1c-4131-bdce-37097b401c75 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:02.795313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c482d-b9c0-4945-8228-d39887f06709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:02.795985 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c482d-b9c0-4945-8228-d39887f06709 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:02.796269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c482d-b9c0-4945-8228-d39887f06709 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:02.796547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c482d-b9c0-4945-8228-d39887f06709 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:02.797291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c482d-b9c0-4945-8228-d39887f06709 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:02.797597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c482d-b9c0-4945-8228-d39887f06709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:02.798993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c482d-b9c0-4945-8228-d39887f06709 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:02.811598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 511/1017] 10.222.0.42 () {56 vars in 1115 bytes} [Fri Jun 3 09:39:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:02.813140 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d1105-4a6d-4846-b953-0fcbf617aa49 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:02.912915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d1105-4a6d-4846-b953-0fcbf617aa49 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:02.947394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b04d8a-a749-4bf3-96b3-56f46348b552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:02.948188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b04d8a-a749-4bf3-96b3-56f46348b552 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:02.948470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b04d8a-a749-4bf3-96b3-56f46348b552 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:02.948745 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b04d8a-a749-4bf3-96b3-56f46348b552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:02.959698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b04d8a-a749-4bf3-96b3-56f46348b552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:02.961149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b04d8a-a749-4bf3-96b3-56f46348b552 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-e2b04d8a-a749-4bf3-96b3-56f46348b552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:39:02.978526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:02.979422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:02.985134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:39:02.985134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:39:02.985134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:02.985134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:02.985134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 507/1018] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:02.986833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c482d-b9c0-4945-8228-d39887f06709 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:03.022436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:03.062974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cc875-6fa3-4169-b8bb-7100bef16401 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:03.063336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cc875-6fa3-4169-b8bb-7100bef16401 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:03.063539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cc875-6fa3-4169-b8bb-7100bef16401 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:03.065046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 508/1019] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:03.070342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0669f5-fc02-4b71-a4d8-efe68456b2c2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:03.070640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0669f5-fc02-4b71-a4d8-efe68456b2c2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:03.070919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0669f5-fc02-4b71-a4d8-efe68456b2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:03.087355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2f0669f5-fc02-4b71-a4d8-efe68456b2c2 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:03.088664 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2f0669f5-fc02-4b71-a4d8-efe68456b2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:03.091207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c482d-b9c0-4945-8228-d39887f06709 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:03.125998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08fb61-aeb9-429e-bcf8-a9c214c10cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:03.126652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08fb61-aeb9-429e-bcf8-a9c214c10cc5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:03.126938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08fb61-aeb9-429e-bcf8-a9c214c10cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:03.127213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08fb61-aeb9-429e-bcf8-a9c214c10cc5 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:03.127928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08fb61-aeb9-429e-bcf8-a9c214c10cc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:03.128230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08fb61-aeb9-429e-bcf8-a9c214c10cc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:03.130223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08fb61-aeb9-429e-bcf8-a9c214c10cc5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:03.139840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:03.141429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 512/1020] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:03.146910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08fb61-aeb9-429e-bcf8-a9c214c10cc5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:03.160369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 509/1021] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:03 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:39:03.183109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0669f5-fc02-4b71-a4d8-efe68456b2c2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:03.243108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:03.249462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08fb61-aeb9-429e-bcf8-a9c214c10cc5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:03.286148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adce52-a13a-4177-8d6d-89d7e8b0a101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:03.287654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adce52-a13a-4177-8d6d-89d7e8b0a101 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:03.288742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adce52-a13a-4177-8d6d-89d7e8b0a101 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:03.289173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adce52-a13a-4177-8d6d-89d7e8b0a101 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:03.290274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0669f5-fc02-4b71-a4d8-efe68456b2c2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:03.297218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4adce52-a13a-4177-8d6d-89d7e8b0a101 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:03.299228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4adce52-a13a-4177-8d6d-89d7e8b0a101 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b4adce52-a13a-4177-8d6d-89d7e8b0a101 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:39:03.318264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:03.319389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:39:03.320541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:03.323028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 513/1022] 10.222.0.42 () {56 vars in 1093 bytes} [Fri Jun 3 09:39:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:03.328582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adce52-a13a-4177-8d6d-89d7e8b0a101 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:03.330146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f53c1-1c77-4f92-b673-3d3f0145fd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:03.330819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f53c1-1c77-4f92-b673-3d3f0145fd8f None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:03.331217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f53c1-1c77-4f92-b673-3d3f0145fd8f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:03.331388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f53c1-1c77-4f92-b673-3d3f0145fd8f None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:03.332092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f53c1-1c77-4f92-b673-3d3f0145fd8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:03.334035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f53c1-1c77-4f92-b673-3d3f0145fd8f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:03.428464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adce52-a13a-4177-8d6d-89d7e8b0a101 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:03.430116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 510/1023] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:39:03 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:39:03.475998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6b507-ccbc-4979-8008-3b1f773c783b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:03.476599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6b507-ccbc-4979-8008-3b1f773c783b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:03.476881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6b507-ccbc-4979-8008-3b1f773c783b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:03.477156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6b507-ccbc-4979-8008-3b1f773c783b None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:03.478039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6b507-ccbc-4979-8008-3b1f773c783b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:03.478349 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6b507-ccbc-4979-8008-3b1f773c783b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:03.479895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6b507-ccbc-4979-8008-3b1f773c783b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:03.505534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 514/1024] 10.222.0.42 () {56 vars in 1133 bytes} [Fri Jun 3 09:39:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:39:03.506956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f53c1-1c77-4f92-b673-3d3f0145fd8f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:03.555228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:03.618586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f53c1-1c77-4f92-b673-3d3f0145fd8f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:03.656286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2436b-108c-4f75-97b6-610dc1afee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:03.656941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2436b-108c-4f75-97b6-610dc1afee31 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:03.657224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2436b-108c-4f75-97b6-610dc1afee31 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:03.657497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2436b-108c-4f75-97b6-610dc1afee31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:03.735406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2436b-108c-4f75-97b6-610dc1afee31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:03.736815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2436b-108c-4f75-97b6-610dc1afee31 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:03.782550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:03.844450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 511/1025] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:39:03.846157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6b507-ccbc-4979-8008-3b1f773c783b None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:03.857399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:03.947964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6b507-ccbc-4979-8008-3b1f773c783b None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:03.982682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf24883-0cfd-4214-b5a3-a20198ef7027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:03.983481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf24883-0cfd-4214-b5a3-a20198ef7027 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:03.983764 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf24883-0cfd-4214-b5a3-a20198ef7027 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:03.984042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf24883-0cfd-4214-b5a3-a20198ef7027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:03.995204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf24883-0cfd-4214-b5a3-a20198ef7027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:03.996583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf24883-0cfd-4214-b5a3-a20198ef7027 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-adf24883-0cfd-4214-b5a3-a20198ef7027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:39:04.014132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:04.015008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:04.015886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:39:04.015886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:39:04.015886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:04.015886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:04.018196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 515/1026] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:04.021612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2436b-108c-4f75-97b6-610dc1afee31 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:04.058579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bd1e5-c4ae-4f29-9666-498e87b9fafb None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:04.058880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bd1e5-c4ae-4f29-9666-498e87b9fafb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:04.059133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bd1e5-c4ae-4f29-9666-498e87b9fafb None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:04.060649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 516/1027] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:39:04.065430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188dd93-d434-42fa-9c85-48aa2c8dcc54 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:04.065717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188dd93-d434-42fa-9c85-48aa2c8dcc54 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:04.066032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188dd93-d434-42fa-9c85-48aa2c8dcc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:04.082985 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3188dd93-d434-42fa-9c85-48aa2c8dcc54 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:04.084174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3188dd93-d434-42fa-9c85-48aa2c8dcc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:04.125790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2436b-108c-4f75-97b6-610dc1afee31 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:04.149025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:04.160081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e1586-f3d1-40ac-bc1a-9fe6c898ed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:04.170856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e1586-f3d1-40ac-bc1a-9fe6c898ed9c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:04.170856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e1586-f3d1-40ac-bc1a-9fe6c898ed9c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:04.170856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e1586-f3d1-40ac-bc1a-9fe6c898ed9c None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:04.170856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e1586-f3d1-40ac-bc1a-9fe6c898ed9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:04.170856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e1586-f3d1-40ac-bc1a-9fe6c898ed9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:04.171309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 517/1028] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:39:04.171397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e1586-f3d1-40ac-bc1a-9fe6c898ed9c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:04.178188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 512/1029] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:39:04.182892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188dd93-d434-42fa-9c85-48aa2c8dcc54 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:04.186304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e1586-f3d1-40ac-bc1a-9fe6c898ed9c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:04.189132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:04.293928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188dd93-d434-42fa-9c85-48aa2c8dcc54 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:04.296897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e1586-f3d1-40ac-bc1a-9fe6c898ed9c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:04.331441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebcf1-33e2-43f6-9d31-3ee95c1dc49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:04.331766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c2ce4-301f-4066-b97b-a3cf2896b2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:04.332335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebcf1-33e2-43f6-9d31-3ee95c1dc49c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:04.332567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c2ce4-301f-4066-b97b-a3cf2896b2cb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:04.332667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebcf1-33e2-43f6-9d31-3ee95c1dc49c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:04.332854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c2ce4-301f-4066-b97b-a3cf2896b2cb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:04.332962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebcf1-33e2-43f6-9d31-3ee95c1dc49c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:04.333136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c2ce4-301f-4066-b97b-a3cf2896b2cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:04.334335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c2ce4-301f-4066-b97b-a3cf2896b2cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:04.335763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c2ce4-301f-4066-b97b-a3cf2896b2cb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:04.341917 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 513/1030] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:04.342065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ebcf1-33e2-43f6-9d31-3ee95c1dc49c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:04.342065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ebcf1-33e2-43f6-9d31-3ee95c1dc49c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:04.343137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c2ce4-301f-4066-b97b-a3cf2896b2cb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-a38ebcf1-33e2-43f6-9d31-3ee95c1dc49c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:39:04.355778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:04.356672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:04.357545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:39:04.357545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:39:04.357545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:39:04.357545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:39:04.357545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:39:04.357545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:39:04.357545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:39:04.357545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:04.362065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 518/1031] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:39:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:04.362582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebcf1-33e2-43f6-9d31-3ee95c1dc49c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:04.415492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:04.426266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:04.458674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c2ce4-301f-4066-b97b-a3cf2896b2cb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:04.468290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebcf1-33e2-43f6-9d31-3ee95c1dc49c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:04.496580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786020f9-da48-45a3-8168-09b58c92a969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:04.497480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786020f9-da48-45a3-8168-09b58c92a969 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:04.497812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786020f9-da48-45a3-8168-09b58c92a969 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:04.498124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786020f9-da48-45a3-8168-09b58c92a969 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:04.498845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786020f9-da48-45a3-8168-09b58c92a969 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:04.500286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786020f9-da48-45a3-8168-09b58c92a969 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:04.504497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed49a0f-2fce-4875-88c7-73d1f37b7e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:04.505483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed49a0f-2fce-4875-88c7-73d1f37b7e9f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:04.505768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed49a0f-2fce-4875-88c7-73d1f37b7e9f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:04.506076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed49a0f-2fce-4875-88c7-73d1f37b7e9f None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:04.506865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed49a0f-2fce-4875-88c7-73d1f37b7e9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:04.507176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed49a0f-2fce-4875-88c7-73d1f37b7e9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:04.510629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed49a0f-2fce-4875-88c7-73d1f37b7e9f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:04.522869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 519/1032] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:39:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:04.528658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed49a0f-2fce-4875-88c7-73d1f37b7e9f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:04.617503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 514/1033] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:39:04 2022] POST /identity/v3/projects => generated 311 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:39:04.641310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed49a0f-2fce-4875-88c7-73d1f37b7e9f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:04.654696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:04.681915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb36c9-0954-4f31-a1b9-c851072aba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:04.682598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb36c9-0954-4f31-a1b9-c851072aba5b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:04.682881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb36c9-0954-4f31-a1b9-c851072aba5b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:04.683155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb36c9-0954-4f31-a1b9-c851072aba5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:04.684008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb36c9-0954-4f31-a1b9-c851072aba5b None admin] RBAC: Adding query filter params (role.id=660489b44e624266a7f8293c906652ed, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=dfe8ef12db914a9693ef470f1706bb5a) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:04.684323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb36c9-0954-4f31-a1b9-c851072aba5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:04.685804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb36c9-0954-4f31-a1b9-c851072aba5b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:04.693062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 520/1034] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:39:04 2022] GET /identity/v3/role_assignments?user.id=dfe8ef12db914a9693ef470f1706bb5a&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=660489b44e624266a7f8293c906652ed => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:39:05.699769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e6206-49cf-4eca-a467-c17d789e5e34 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:05.700317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e6206-49cf-4eca-a467-c17d789e5e34 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:05.700317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e6206-49cf-4eca-a467-c17d789e5e34 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:05.701825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 515/1035] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:05.706760 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad0d8d-58e5-4790-ae72-490217971484 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:05.707136 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad0d8d-58e5-4790-ae72-490217971484 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:05.707901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad0d8d-58e5-4790-ae72-490217971484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:05.724765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b1ad0d8d-58e5-4790-ae72-490217971484 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:05.725956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b1ad0d8d-58e5-4790-ae72-490217971484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:05.795099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 521/1036] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:05.817334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e6206-49cf-4eca-a467-c17d789e5e34 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:05.922282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e6206-49cf-4eca-a467-c17d789e5e34 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:05.945056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:05.957102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ad50b-fddd-4b4c-8b7b-95a326762292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:05.957866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ad50b-fddd-4b4c-8b7b-95a326762292 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:05.958176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ad50b-fddd-4b4c-8b7b-95a326762292 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:05.958567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ad50b-fddd-4b4c-8b7b-95a326762292 None admin] PATH_INFO: `/v3/services/image` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:05.959103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ad50b-fddd-4b4c-8b7b-95a326762292 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:05.960500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ad50b-fddd-4b4c-8b7b-95a326762292 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3a9ad50b-fddd-4b4c-8b7b-95a326762292 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:05.972263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:05.973751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:05.974693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 516/1037] 10.222.0.42 () {56 vars in 1105 bytes} [Fri Jun 3 09:39:05 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:05.981774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad0d8d-58e5-4790-ae72-490217971484 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:06.080251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad0d8d-58e5-4790-ae72-490217971484 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:06.083069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:06.113276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64795532-0740-40f8-be47-31c076f1b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:06.114465 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64795532-0740-40f8-be47-31c076f1b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:06.114746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64795532-0740-40f8-be47-31c076f1b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:06.115066 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64795532-0740-40f8-be47-31c076f1b4d5 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:06.115754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64795532-0740-40f8-be47-31c076f1b4d5 None admin] RBAC: Adding query filter params (name=image) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:06.116059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64795532-0740-40f8-be47-31c076f1b4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:06.117409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64795532-0740-40f8-be47-31c076f1b4d5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:06.123166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 522/1038] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:06.127742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ad50b-fddd-4b4c-8b7b-95a326762292 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:06.228902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ad50b-fddd-4b4c-8b7b-95a326762292 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:06.261788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519389a5-f52b-42b7-8b48-820c2a927b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:06.262476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519389a5-f52b-42b7-8b48-820c2a927b1d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:06.262760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519389a5-f52b-42b7-8b48-820c2a927b1d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:06.263032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519389a5-f52b-42b7-8b48-820c2a927b1d None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:06.263730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519389a5-f52b-42b7-8b48-820c2a927b1d None admin] RBAC: Adding query filter params (type=image) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:06.264033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519389a5-f52b-42b7-8b48-820c2a927b1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:06.265390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519389a5-f52b-42b7-8b48-820c2a927b1d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:06.275554 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 517/1039] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:06.276807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64795532-0740-40f8-be47-31c076f1b4d5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:06.383394 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64795532-0740-40f8-be47-31c076f1b4d5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:06.418271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab597c4e-306f-4972-9e11-8ee13d1ac3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:06.418931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab597c4e-306f-4972-9e11-8ee13d1ac3b0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:06.419212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab597c4e-306f-4972-9e11-8ee13d1ac3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:06.419534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab597c4e-306f-4972-9e11-8ee13d1ac3b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:06.420316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab597c4e-306f-4972-9e11-8ee13d1ac3b0 None admin] RBAC: Adding query filter params (interface=public, service_id=93d4e50597fa4f70976bbf3ddbb567a3, region_id=RegionOne) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:06.420625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab597c4e-306f-4972-9e11-8ee13d1ac3b0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:06.422293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab597c4e-306f-4972-9e11-8ee13d1ac3b0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:06.431845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 523/1040] 10.222.0.42 () {56 vars in 1251 bytes} [Fri Jun 3 09:39:06 2022] GET /identity/v3/endpoints?service_id=93d4e50597fa4f70976bbf3ddbb567a3&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:06.433010 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519389a5-f52b-42b7-8b48-820c2a927b1d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:06.484844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:06.536287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519389a5-f52b-42b7-8b48-820c2a927b1d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:06.570008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f8d69-4d19-4d13-8acf-b7195e2e935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:06.570677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f8d69-4d19-4d13-8acf-b7195e2e935a None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:06.570961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f8d69-4d19-4d13-8acf-b7195e2e935a None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:06.571242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f8d69-4d19-4d13-8acf-b7195e2e935a None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:06.571916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f8d69-4d19-4d13-8acf-b7195e2e935a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:06.573452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f8d69-4d19-4d13-8acf-b7195e2e935a None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:06.585978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 518/1041] 10.222.0.42 () {56 vars in 1087 bytes} [Fri Jun 3 09:39:06 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:39:06.954551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e5d3c-3ec4-4800-bcc3-71a60b32d187 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:06.955099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e5d3c-3ec4-4800-bcc3-71a60b32d187 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:06.955099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e5d3c-3ec4-4800-bcc3-71a60b32d187 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:06.956604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 524/1042] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:06.962437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b37422-92e4-4bb4-b8a1-a0d1785ba42a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:06.962833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b37422-92e4-4bb4-b8a1-a0d1785ba42a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:06.963015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b37422-92e4-4bb4-b8a1-a0d1785ba42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:06.971522 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058be3-0fbf-404b-9f68-a7c0c745d55e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:06.971812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058be3-0fbf-404b-9f68-a7c0c745d55e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:06.972095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058be3-0fbf-404b-9f68-a7c0c745d55e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:06.973623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 525/1043] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:39:06.978469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e89a35-ec11-4eb5-9a2a-4f192c930fc5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:06.978764 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e89a35-ec11-4eb5-9a2a-4f192c930fc5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:06.979054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e89a35-ec11-4eb5-9a2a-4f192c930fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:06.980222 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b4b37422-92e4-4bb4-b8a1-a0d1785ba42a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:06.981436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b4b37422-92e4-4bb4-b8a1-a0d1785ba42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:06.995958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-68e89a35-ec11-4eb5-9a2a-4f192c930fc5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:06.997044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-68e89a35-ec11-4eb5-9a2a-4f192c930fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:07.023152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:07.057749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 519/1044] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:07.073928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 526/1045] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:39:07.082961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b37422-92e4-4bb4-b8a1-a0d1785ba42a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:07.092875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e89a35-ec11-4eb5-9a2a-4f192c930fc5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:07.133490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:07.189954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b37422-92e4-4bb4-b8a1-a0d1785ba42a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:07.196172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e89a35-ec11-4eb5-9a2a-4f192c930fc5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:07.225659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf7504-fed6-40df-84b6-e19b758f480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:07.226411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf7504-fed6-40df-84b6-e19b758f480e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:07.226694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf7504-fed6-40df-84b6-e19b758f480e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:07.226968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf7504-fed6-40df-84b6-e19b758f480e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:07.228122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf7504-fed6-40df-84b6-e19b758f480e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:07.229858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf7504-fed6-40df-84b6-e19b758f480e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:07.230828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20265055-8c74-452e-97f1-9bc8bbf91bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:07.231664 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 520/1046] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:07.231772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20265055-8c74-452e-97f1-9bc8bbf91bb3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:07.231995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20265055-8c74-452e-97f1-9bc8bbf91bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:07.232298 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20265055-8c74-452e-97f1-9bc8bbf91bb3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:07.237221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf7504-fed6-40df-84b6-e19b758f480e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:07.243528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20265055-8c74-452e-97f1-9bc8bbf91bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-20265055-8c74-452e-97f1-9bc8bbf91bb3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:07.251564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:07.252501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:07.253343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 527/1047] 10.222.0.42 () {56 vars in 1120 bytes} [Fri Jun 3 09:39:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:07.259270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20265055-8c74-452e-97f1-9bc8bbf91bb3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:07.286055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:07.346475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf7504-fed6-40df-84b6-e19b758f480e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:07.360679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:07.365114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20265055-8c74-452e-97f1-9bc8bbf91bb3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:07.382274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4157e-f50c-48e3-ba07-e3da2837a345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:07.383030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4157e-f50c-48e3-ba07-e3da2837a345 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:07.383319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4157e-f50c-48e3-ba07-e3da2837a345 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:07.383599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4157e-f50c-48e3-ba07-e3da2837a345 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:07.384352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4157e-f50c-48e3-ba07-e3da2837a345 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:07.385798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed4157e-f50c-48e3-ba07-e3da2837a345 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:07.401103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d51f4d-0176-4a5a-86ed-7b8ec2d256f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:07.401822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d51f4d-0176-4a5a-86ed-7b8ec2d256f9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:07.402187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d51f4d-0176-4a5a-86ed-7b8ec2d256f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:07.402457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d51f4d-0176-4a5a-86ed-7b8ec2d256f9 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:07.403197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d51f4d-0176-4a5a-86ed-7b8ec2d256f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:07.407367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d51f4d-0176-4a5a-86ed-7b8ec2d256f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:07.407367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d51f4d-0176-4a5a-86ed-7b8ec2d256f9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:07.413822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 528/1048] 10.222.0.42 () {56 vars in 1115 bytes} [Fri Jun 3 09:39:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:39:07.419239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d51f4d-0176-4a5a-86ed-7b8ec2d256f9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:07.498013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 521/1049] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:39:07 2022] POST /identity/v3/users => generated 285 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:39:07.507203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:07.524678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d51f4d-0176-4a5a-86ed-7b8ec2d256f9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:07.561377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfeb5b-6f4d-488f-b92a-01e2b4ac244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:07.562224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfeb5b-6f4d-488f-b92a-01e2b4ac244d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:07.562506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfeb5b-6f4d-488f-b92a-01e2b4ac244d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:07.562783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfeb5b-6f4d-488f-b92a-01e2b4ac244d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:07.574250 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfeb5b-6f4d-488f-b92a-01e2b4ac244d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:07.575779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfeb5b-6f4d-488f-b92a-01e2b4ac244d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-c5bfeb5b-6f4d-488f-b92a-01e2b4ac244d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:39:07.595745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:07.596818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:07.597732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:39:07.597732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:39:07.597732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:07.597732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:07.599280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 529/1050] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:07.603539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4157e-f50c-48e3-ba07-e3da2837a345 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:07.717962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4157e-f50c-48e3-ba07-e3da2837a345 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:07.753383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f62558-1c16-4b07-8b11-a8c9c0461bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:07.754067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f62558-1c16-4b07-8b11-a8c9c0461bfa None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:07.754355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f62558-1c16-4b07-8b11-a8c9c0461bfa None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:07.754630 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f62558-1c16-4b07-8b11-a8c9c0461bfa None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:07.755352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f62558-1c16-4b07-8b11-a8c9c0461bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:07.755655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f62558-1c16-4b07-8b11-a8c9c0461bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:07.757035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f62558-1c16-4b07-8b11-a8c9c0461bfa None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:07.771134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 522/1051] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:07.776974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfeb5b-6f4d-488f-b92a-01e2b4ac244d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:07.877804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfeb5b-6f4d-488f-b92a-01e2b4ac244d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:07.911566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1c94-ac6c-4e91-938b-69692d9f318c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:07.912343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1c94-ac6c-4e91-938b-69692d9f318c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:07.912613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1c94-ac6c-4e91-938b-69692d9f318c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:07.912886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1c94-ac6c-4e91-938b-69692d9f318c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:07.915874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:07.921086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf1c94-ac6c-4e91-938b-69692d9f318c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:07.922547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf1c94-ac6c-4e91-938b-69692d9f318c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-55bf1c94-ac6c-4e91-938b-69692d9f318c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:39:07.937365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:07.938515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:07.939498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 530/1052] 10.222.0.42 () {56 vars in 1093 bytes} [Fri Jun 3 09:39:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:07.944412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f62558-1c16-4b07-8b11-a8c9c0461bfa None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:07.973525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:08.045919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f62558-1c16-4b07-8b11-a8c9c0461bfa None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:08.080575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22755cf7-bb65-4252-a1b9-c506c658faff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:08.081256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22755cf7-bb65-4252-a1b9-c506c658faff None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:08.081538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22755cf7-bb65-4252-a1b9-c506c658faff None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:08.081816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22755cf7-bb65-4252-a1b9-c506c658faff None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:08.082758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22755cf7-bb65-4252-a1b9-c506c658faff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:08.083070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22755cf7-bb65-4252-a1b9-c506c658faff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:08.084506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22755cf7-bb65-4252-a1b9-c506c658faff None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:08.106112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 523/1053] 10.222.0.42 () {56 vars in 1133 bytes} [Fri Jun 3 09:39:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:08.107383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1c94-ac6c-4e91-938b-69692d9f318c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:08.207908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1c94-ac6c-4e91-938b-69692d9f318c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:08.241428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a402fc8-dda4-4420-b2dd-9ca1afd29f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:08.242254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a402fc8-dda4-4420-b2dd-9ca1afd29f2d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:08.242535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a402fc8-dda4-4420-b2dd-9ca1afd29f2d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:08.242809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a402fc8-dda4-4420-b2dd-9ca1afd29f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:08.252884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a402fc8-dda4-4420-b2dd-9ca1afd29f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:08.254882 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a402fc8-dda4-4420-b2dd-9ca1afd29f2d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-1a402fc8-dda4-4420-b2dd-9ca1afd29f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:39:08.272251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:08.273213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:08.274523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:39:08.274523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:39:08.274523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:08.274523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:08.278063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 531/1054] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:08.279044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22755cf7-bb65-4252-a1b9-c506c658faff None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:08.361994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:08.377234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22755cf7-bb65-4252-a1b9-c506c658faff None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:08.411592 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c698fe5-9d9b-44a0-811d-8e084249041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:08.412254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c698fe5-9d9b-44a0-811d-8e084249041e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:08.412532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c698fe5-9d9b-44a0-811d-8e084249041e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:08.412802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c698fe5-9d9b-44a0-811d-8e084249041e None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:08.413782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c698fe5-9d9b-44a0-811d-8e084249041e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:08.414112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c698fe5-9d9b-44a0-811d-8e084249041e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:08.415469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c698fe5-9d9b-44a0-811d-8e084249041e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:08.431591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 524/1055] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:08.432837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a402fc8-dda4-4420-b2dd-9ca1afd29f2d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:08.485018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:08.540370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a402fc8-dda4-4420-b2dd-9ca1afd29f2d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:08.574955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f42fe-1dd4-485a-a313-9b219c7c47bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:08.575744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f42fe-1dd4-485a-a313-9b219c7c47bc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:08.576025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f42fe-1dd4-485a-a313-9b219c7c47bc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:08.576298 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f42fe-1dd4-485a-a313-9b219c7c47bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:08.583441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f42fe-1dd4-485a-a313-9b219c7c47bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:08.584862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f42fe-1dd4-485a-a313-9b219c7c47bc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-d46f42fe-1dd4-485a-a313-9b219c7c47bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:39:08.598396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:08.599259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:08.600143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:39:08.600143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:39:08.600143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:39:08.600143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:39:08.600143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:39:08.600143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:39:08.600143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:39:08.600143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:08.601969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 532/1056] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:08.605309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c698fe5-9d9b-44a0-811d-8e084249041e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:08.709199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c698fe5-9d9b-44a0-811d-8e084249041e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:08.747055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ae460-d224-4267-b13b-67cad8ad351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:08.747710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ae460-d224-4267-b13b-67cad8ad351e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:08.747990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ae460-d224-4267-b13b-67cad8ad351e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:08.748264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ae460-d224-4267-b13b-67cad8ad351e None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:08.749032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ae460-d224-4267-b13b-67cad8ad351e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:08.749336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ae460-d224-4267-b13b-67cad8ad351e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:08.750782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ae460-d224-4267-b13b-67cad8ad351e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:08.767582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 525/1057] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:08.768713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f42fe-1dd4-485a-a313-9b219c7c47bc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:08.860775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ea9ac-b03c-4343-8114-700f125e36ca None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:08.861097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ea9ac-b03c-4343-8114-700f125e36ca None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:08.861331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ea9ac-b03c-4343-8114-700f125e36ca None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:08.862842 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 526/1058] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:08.867778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1f32d-1567-49db-8404-56c8a77e3a9d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:08.868065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1f32d-1567-49db-8404-56c8a77e3a9d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:08.868345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1f32d-1567-49db-8404-56c8a77e3a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:08.869652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f42fe-1dd4-485a-a313-9b219c7c47bc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:08.878753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:08.879422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:08.885604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-58a1f32d-1567-49db-8404-56c8a77e3a9d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:08.886818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-58a1f32d-1567-49db-8404-56c8a77e3a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:08.905211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4fb38-8a95-4e3b-bf92-767106613653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:08.906137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4fb38-8a95-4e3b-bf92-767106613653 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:08.906421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4fb38-8a95-4e3b-bf92-767106613653 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:08.906703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4fb38-8a95-4e3b-bf92-767106613653 None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/dfe8ef12db914a9693ef470f1706bb5a/roles/660489b44e624266a7f8293c906652ed` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:08.908868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4fb38-8a95-4e3b-bf92-767106613653 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=dfe8ef12db914a9693ef470f1706bb5a, role_id=660489b44e624266a7f8293c906652ed)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:08.910326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4fb38-8a95-4e3b-bf92-767106613653 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:08.952014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 527/1059] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:39:08.975378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1f32d-1567-49db-8404-56c8a77e3a9d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:09.009404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 533/1060] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:39:08 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/dfe8ef12db914a9693ef470f1706bb5a/roles/660489b44e624266a7f8293c906652ed => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:39:09.081681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1f32d-1567-49db-8404-56c8a77e3a9d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:09.119958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bcde8-227a-4c80-883b-9d0b69f4d118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:09.121075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bcde8-227a-4c80-883b-9d0b69f4d118 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:09.121511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bcde8-227a-4c80-883b-9d0b69f4d118 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:09.121985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bcde8-227a-4c80-883b-9d0b69f4d118 None admin] PATH_INFO: `/v3/services/image` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:09.122817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bcde8-227a-4c80-883b-9d0b69f4d118 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:09.124452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bcde8-227a-4c80-883b-9d0b69f4d118 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:09.127776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-965bcde8-227a-4c80-883b-9d0b69f4d118 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:09.138871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:09.146145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:09.146973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 528/1061] 10.222.0.42 () {56 vars in 1105 bytes} [Fri Jun 3 09:39:08 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:09.147760 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4fb38-8a95-4e3b-bf92-767106613653 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:09.258987 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4fb38-8a95-4e3b-bf92-767106613653 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:09.292024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41613975-a43a-4944-aebf-f23f6d203535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:09.292695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41613975-a43a-4944-aebf-f23f6d203535 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:09.292979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41613975-a43a-4944-aebf-f23f6d203535 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:09.293266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41613975-a43a-4944-aebf-f23f6d203535 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:09.294011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41613975-a43a-4944-aebf-f23f6d203535 None admin] RBAC: Adding query filter params (name=image) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:09.294325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41613975-a43a-4944-aebf-f23f6d203535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:09.295700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41613975-a43a-4944-aebf-f23f6d203535 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:09.305527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 534/1062] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:09.306700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bcde8-227a-4c80-883b-9d0b69f4d118 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:09.407386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bcde8-227a-4c80-883b-9d0b69f4d118 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:09.441769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bacedb-2e39-445a-bc7d-81c0cb0d15fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:09.442462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bacedb-2e39-445a-bc7d-81c0cb0d15fb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:09.442749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bacedb-2e39-445a-bc7d-81c0cb0d15fb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:09.443031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bacedb-2e39-445a-bc7d-81c0cb0d15fb None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:09.443750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bacedb-2e39-445a-bc7d-81c0cb0d15fb None admin] RBAC: Adding query filter params (type=image) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:09.444043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bacedb-2e39-445a-bc7d-81c0cb0d15fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:09.445411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bacedb-2e39-445a-bc7d-81c0cb0d15fb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:09.454760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 529/1063] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:39:09.455932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41613975-a43a-4944-aebf-f23f6d203535 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:09.488903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:09.562888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41613975-a43a-4944-aebf-f23f6d203535 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:09.596754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2c969-6e45-40f9-bdef-ae4d47d40b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:09.597410 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2c969-6e45-40f9-bdef-ae4d47d40b4d None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:09.598240 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2c969-6e45-40f9-bdef-ae4d47d40b4d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:09.598518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2c969-6e45-40f9-bdef-ae4d47d40b4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:09.599244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2c969-6e45-40f9-bdef-ae4d47d40b4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:09.600598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2c969-6e45-40f9-bdef-ae4d47d40b4d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:09.671984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 535/1064] 10.222.0.42 () {60 vars in 1144 bytes} [Fri Jun 3 09:39:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:39:09.847318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc4f38-6fba-42da-8850-a2b672a80795 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:09.847708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc4f38-6fba-42da-8850-a2b672a80795 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:09.848465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc4f38-6fba-42da-8850-a2b672a80795 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:09.850148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 530/1065] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:09.855854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930c886-6b41-436c-9c60-073ccb6e93bf None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:09.856150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930c886-6b41-436c-9c60-073ccb6e93bf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:09.856437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930c886-6b41-436c-9c60-073ccb6e93bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:09.873778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-8930c886-6b41-436c-9c60-073ccb6e93bf None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:09.875019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8930c886-6b41-436c-9c60-073ccb6e93bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:09.949339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 536/1066] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:09.972177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc4f38-6fba-42da-8850-a2b672a80795 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:10.035390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:10.074697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc4f38-6fba-42da-8850-a2b672a80795 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:10.107731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c5d7a-e483-41e1-9697-679b9a00c6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:10.108438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c5d7a-e483-41e1-9697-679b9a00c6c6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:10.108717 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c5d7a-e483-41e1-9697-679b9a00c6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:10.108988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c5d7a-e483-41e1-9697-679b9a00c6c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:10.119704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c5d7a-e483-41e1-9697-679b9a00c6c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-2c3c5d7a-e483-41e1-9697-679b9a00c6c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:10.127341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:10.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:10.129254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:10.133771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 531/1067] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:10.134854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930c886-6b41-436c-9c60-073ccb6e93bf None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:10.151125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:10.237044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930c886-6b41-436c-9c60-073ccb6e93bf None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:10.271739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ed0de-efd3-4c10-9570-3cf93041cfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:10.272409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ed0de-efd3-4c10-9570-3cf93041cfa9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:10.272690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ed0de-efd3-4c10-9570-3cf93041cfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:10.273017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ed0de-efd3-4c10-9570-3cf93041cfa9 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:10.273758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ed0de-efd3-4c10-9570-3cf93041cfa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:10.274088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ed0de-efd3-4c10-9570-3cf93041cfa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:10.275462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ed0de-efd3-4c10-9570-3cf93041cfa9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:10.283928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 537/1068] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:39:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:10.289665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c5d7a-e483-41e1-9697-679b9a00c6c6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:10.387106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c5d7a-e483-41e1-9697-679b9a00c6c6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:10.409763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:10.420645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765e02c-c3ce-4ee6-985f-2dbb45703e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:10.424530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765e02c-c3ce-4ee6-985f-2dbb45703e84 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:10.424530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765e02c-c3ce-4ee6-985f-2dbb45703e84 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:10.424530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765e02c-c3ce-4ee6-985f-2dbb45703e84 None admin] PATH_INFO: `/v3/users/1b7b3b49edac4c18a7a3eb1f3f9b6304` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:10.435041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765e02c-c3ce-4ee6-985f-2dbb45703e84 None admin] RBAC: Authorizing `identity:get_user(user_id=1b7b3b49edac4c18a7a3eb1f3f9b6304)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:10.436531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765e02c-c3ce-4ee6-985f-2dbb45703e84 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:10.438501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 532/1069] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:10 2022] GET /identity/v3/users/1b7b3b49edac4c18a7a3eb1f3f9b6304 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:10.444442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ed0de-efd3-4c10-9570-3cf93041cfa9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:10.524371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:10.546922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ed0de-efd3-4c10-9570-3cf93041cfa9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:10.582745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25237930-8f34-4410-ad65-4609e87d8719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:10.583398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25237930-8f34-4410-ad65-4609e87d8719 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:10.583675 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25237930-8f34-4410-ad65-4609e87d8719 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:10.583942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25237930-8f34-4410-ad65-4609e87d8719 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:10.651884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25237930-8f34-4410-ad65-4609e87d8719 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:10.653305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25237930-8f34-4410-ad65-4609e87d8719 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:10.733538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:10.757854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 538/1070] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:39:10.763685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765e02c-c3ce-4ee6-985f-2dbb45703e84 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:10.868898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765e02c-c3ce-4ee6-985f-2dbb45703e84 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:10.905646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f3004-2c80-45c1-adeb-84ab8aebc977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:10.906479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3004-2c80-45c1-adeb-84ab8aebc977 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:10.906739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3004-2c80-45c1-adeb-84ab8aebc977 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:10.907015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3004-2c80-45c1-adeb-84ab8aebc977 None admin] PATH_INFO: `/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:10.908180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f3004-2c80-45c1-adeb-84ab8aebc977 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb90e04b5f2403e9c622416f7e4cefb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:10.909626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f3004-2c80-45c1-adeb-84ab8aebc977 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:10.911900 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 533/1071] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:10 2022] GET /identity/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:10.918332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25237930-8f34-4410-ad65-4609e87d8719 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:11.022294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25237930-8f34-4410-ad65-4609e87d8719 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:11.056141 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf61abb-7ba1-4af2-8210-0228a5824aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:11.056794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf61abb-7ba1-4af2-8210-0228a5824aff None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:11.057081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf61abb-7ba1-4af2-8210-0228a5824aff None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:11.057359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf61abb-7ba1-4af2-8210-0228a5824aff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:11.058242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf61abb-7ba1-4af2-8210-0228a5824aff None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=eeb90e04b5f2403e9c622416f7e4cefb, user.id=1b7b3b49edac4c18a7a3eb1f3f9b6304) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:11.058558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf61abb-7ba1-4af2-8210-0228a5824aff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:11.059981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf61abb-7ba1-4af2-8210-0228a5824aff None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:11.072206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 539/1072] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:39:10 2022] GET /identity/v3/role_assignments?user.id=1b7b3b49edac4c18a7a3eb1f3f9b6304&scope.project.id=eeb90e04b5f2403e9c622416f7e4cefb&role.id=fe4d636f5c4842f18569347911e6cdec => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:11.281861 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702371d-b1a2-4718-b4f0-f802725143f4 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:11.282389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702371d-b1a2-4718-b4f0-f802725143f4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:11.282389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702371d-b1a2-4718-b4f0-f802725143f4 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:11.283855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 534/1073] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:11.289083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fd7ba-3a83-40d6-b054-e7ff5d3c4ced None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:11.289479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fd7ba-3a83-40d6-b054-e7ff5d3c4ced None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:11.289640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fd7ba-3a83-40d6-b054-e7ff5d3c4ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:11.306602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-290fd7ba-3a83-40d6-b054-e7ff5d3c4ced None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:11.307765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-290fd7ba-3a83-40d6-b054-e7ff5d3c4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:11.341797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:11.374581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 540/1074] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:11.397941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702371d-b1a2-4718-b4f0-f802725143f4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:11.451063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:11.501442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702371d-b1a2-4718-b4f0-f802725143f4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:11.535919 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcf046-4987-4192-bf96-536214ae09eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:11.536661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcf046-4987-4192-bf96-536214ae09eb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:11.536947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcf046-4987-4192-bf96-536214ae09eb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:11.537228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcf046-4987-4192-bf96-536214ae09eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:11.549313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcf046-4987-4192-bf96-536214ae09eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-63fcf046-4987-4192-bf96-536214ae09eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:11.558200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:11.559166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:11.560081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 535/1075] 10.222.0.42 () {56 vars in 1120 bytes} [Fri Jun 3 09:39:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:11.567321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fd7ba-3a83-40d6-b054-e7ff5d3c4ced None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:11.669607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fd7ba-3a83-40d6-b054-e7ff5d3c4ced None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:11.704205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b006189-3f3e-4172-99c2-c98a56829a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:11.704873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b006189-3f3e-4172-99c2-c98a56829a84 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:11.705160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b006189-3f3e-4172-99c2-c98a56829a84 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:11.705439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b006189-3f3e-4172-99c2-c98a56829a84 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:11.706217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b006189-3f3e-4172-99c2-c98a56829a84 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:11.706528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b006189-3f3e-4172-99c2-c98a56829a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:11.708840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b006189-3f3e-4172-99c2-c98a56829a84 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:11.728990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 541/1076] 10.222.0.42 () {56 vars in 1115 bytes} [Fri Jun 3 09:39:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:39:11.730229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcf046-4987-4192-bf96-536214ae09eb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:11.830845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcf046-4987-4192-bf96-536214ae09eb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:11.841157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:11.865295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7f217-f336-4540-9df1-cbfc6e7efc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:11.866380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7f217-f336-4540-9df1-cbfc6e7efc16 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:11.866818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7f217-f336-4540-9df1-cbfc6e7efc16 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:11.867252 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7f217-f336-4540-9df1-cbfc6e7efc16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:11.878897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7f217-f336-4540-9df1-cbfc6e7efc16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:11.880559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7f217-f336-4540-9df1-cbfc6e7efc16 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-6de7f217-f336-4540-9df1-cbfc6e7efc16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:39:11.897728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:11.898804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:11.899765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:39:11.899765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:39:11.899765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:11.899765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:11.900030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 536/1077] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:11.906051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b006189-3f3e-4172-99c2-c98a56829a84 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:11.940266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:11.975627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408d0e5-0959-4778-8314-5d73ab1a42dc None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:11.975911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408d0e5-0959-4778-8314-5d73ab1a42dc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:11.976194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408d0e5-0959-4778-8314-5d73ab1a42dc None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:11.977932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 537/1078] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:39:11.982685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a6202-3d60-433c-82bd-220fdfccb12e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:11.982976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a6202-3d60-433c-82bd-220fdfccb12e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:11.983258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a6202-3d60-433c-82bd-220fdfccb12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:12.000783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cd8a6202-3d60-433c-82bd-220fdfccb12e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:12.002007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cd8a6202-3d60-433c-82bd-220fdfccb12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:12.011907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b006189-3f3e-4172-99c2-c98a56829a84 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:12.046720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81d515-f699-447c-b418-a244a00e7fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:12.047370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81d515-f699-447c-b418-a244a00e7fad None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:12.047657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81d515-f699-447c-b418-a244a00e7fad None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:12.047934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81d515-f699-447c-b418-a244a00e7fad None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:12.048653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81d515-f699-447c-b418-a244a00e7fad None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:12.048959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81d515-f699-447c-b418-a244a00e7fad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:12.050350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81d515-f699-447c-b418-a244a00e7fad None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:12.062202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 542/1079] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:12.067648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81d515-f699-447c-b418-a244a00e7fad None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:12.072618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 538/1080] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:12.094907 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a6202-3d60-433c-82bd-220fdfccb12e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:12.165353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:12.175492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81d515-f699-447c-b418-a244a00e7fad None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:12.190407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:12.198889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a6202-3d60-433c-82bd-220fdfccb12e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:12.214560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d21864-7180-4737-aa6a-8b4692293a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:12.215353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d21864-7180-4737-aa6a-8b4692293a60 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:12.215635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d21864-7180-4737-aa6a-8b4692293a60 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:12.215912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d21864-7180-4737-aa6a-8b4692293a60 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:12.223447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d21864-7180-4737-aa6a-8b4692293a60 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:12.224881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d21864-7180-4737-aa6a-8b4692293a60 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:12.236292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd505924-8388-451f-ac47-6155b369f2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:12.236943 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd505924-8388-451f-ac47-6155b369f2e4 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:12.237230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd505924-8388-451f-ac47-6155b369f2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:12.237509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd505924-8388-451f-ac47-6155b369f2e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-20d21864-7180-4737-aa6a-8b4692293a60 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:39:12.239618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:12.240576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:12.241520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 543/1081] 10.222.0.42 () {56 vars in 1093 bytes} [Fri Jun 3 09:39:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:12.246482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d21864-7180-4737-aa6a-8b4692293a60 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:12.316248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd505924-8388-451f-ac47-6155b369f2e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:12.317754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd505924-8388-451f-ac47-6155b369f2e4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:12.348998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d21864-7180-4737-aa6a-8b4692293a60 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:12.386007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d67496-66d0-40ce-831c-49a381daf929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:12.386744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d67496-66d0-40ce-831c-49a381daf929 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:12.387027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d67496-66d0-40ce-831c-49a381daf929 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:12.387302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d67496-66d0-40ce-831c-49a381daf929 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:12.388122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d67496-66d0-40ce-831c-49a381daf929 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:12.388427 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d67496-66d0-40ce-831c-49a381daf929 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:12.389844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d67496-66d0-40ce-831c-49a381daf929 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:12.407686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:12.409855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 544/1082] 10.222.0.42 () {56 vars in 1133 bytes} [Fri Jun 3 09:39:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:12.415482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d67496-66d0-40ce-831c-49a381daf929 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:12.426311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 539/1083] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 09:39:12.427282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd505924-8388-451f-ac47-6155b369f2e4 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:12.430672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:12.522357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d67496-66d0-40ce-831c-49a381daf929 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:12.528212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd505924-8388-451f-ac47-6155b369f2e4 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:12.556817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd08e5b-5051-4b97-9e1d-8d4a3fae0443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:12.557758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd08e5b-5051-4b97-9e1d-8d4a3fae0443 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:12.558227 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd08e5b-5051-4b97-9e1d-8d4a3fae0443 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:12.558660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd08e5b-5051-4b97-9e1d-8d4a3fae0443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:12.563244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11ab9e-61cd-4308-9790-92923e21feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:12.564030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11ab9e-61cd-4308-9790-92923e21feed None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:12.564311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11ab9e-61cd-4308-9790-92923e21feed None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:12.564583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11ab9e-61cd-4308-9790-92923e21feed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:12.575457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11ab9e-61cd-4308-9790-92923e21feed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:12.576944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11ab9e-61cd-4308-9790-92923e21feed None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8e11ab9e-61cd-4308-9790-92923e21feed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:39:12.595724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:12.596822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:12.597768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:39:12.597768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:39:12.597768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:12.597768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:12.597768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 540/1084] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:12.602651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11ab9e-61cd-4308-9790-92923e21feed None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:12.634385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd08e5b-5051-4b97-9e1d-8d4a3fae0443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:12.635844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd08e5b-5051-4b97-9e1d-8d4a3fae0443 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:12.648864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:12.657698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:12.710740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11ab9e-61cd-4308-9790-92923e21feed None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:12.751550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 545/1085] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 09:39:12.751747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704af39e-739a-4672-8d86-c18ff7d808bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:12.751747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704af39e-739a-4672-8d86-c18ff7d808bf None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:12.751747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704af39e-739a-4672-8d86-c18ff7d808bf None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:12.751747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704af39e-739a-4672-8d86-c18ff7d808bf None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:12.752069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704af39e-739a-4672-8d86-c18ff7d808bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:12.752069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704af39e-739a-4672-8d86-c18ff7d808bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:12.752069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704af39e-739a-4672-8d86-c18ff7d808bf None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:12.752629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd08e5b-5051-4b97-9e1d-8d4a3fae0443 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:12.764283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 541/1086] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:39:12.861246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd08e5b-5051-4b97-9e1d-8d4a3fae0443 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:12.873853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:12.899148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b33ce-63e0-4498-82ab-22dca97b13b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:12.900257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b33ce-63e0-4498-82ab-22dca97b13b6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:12.900690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b33ce-63e0-4498-82ab-22dca97b13b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:12.901125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b33ce-63e0-4498-82ab-22dca97b13b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:12.913855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b33ce-63e0-4498-82ab-22dca97b13b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:12.915607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b33ce-63e0-4498-82ab-22dca97b13b6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-106b33ce-63e0-4498-82ab-22dca97b13b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:39:12.935460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:12.936511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:12.942942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:39:12.942942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:39:12.942942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:39:12.942942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:12.942942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 546/1087] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:12.944532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704af39e-739a-4672-8d86-c18ff7d808bf None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:13.059769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704af39e-739a-4672-8d86-c18ff7d808bf None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:13.063734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:13.096682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23ffbb-0b08-4e85-bb0d-90d8149052be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:13.097372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23ffbb-0b08-4e85-bb0d-90d8149052be None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:13.098274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23ffbb-0b08-4e85-bb0d-90d8149052be None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:13.098552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23ffbb-0b08-4e85-bb0d-90d8149052be None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:13.099268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23ffbb-0b08-4e85-bb0d-90d8149052be None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:13.099574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23ffbb-0b08-4e85-bb0d-90d8149052be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:13.100939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23ffbb-0b08-4e85-bb0d-90d8149052be None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:13.113579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 542/1088] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:13.119031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b33ce-63e0-4498-82ab-22dca97b13b6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:13.217568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b33ce-63e0-4498-82ab-22dca97b13b6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:13.250851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48f3c2-98f6-44f5-a36d-296c0d1df691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:13.251635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48f3c2-98f6-44f5-a36d-296c0d1df691 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:13.251916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48f3c2-98f6-44f5-a36d-296c0d1df691 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:13.252188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48f3c2-98f6-44f5-a36d-296c0d1df691 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:13.259273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48f3c2-98f6-44f5-a36d-296c0d1df691 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:13.260704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48f3c2-98f6-44f5-a36d-296c0d1df691 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-bd48f3c2-98f6-44f5-a36d-296c0d1df691 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:39:13.274654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:13.275517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:13.276404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:39:13.276404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:39:13.276404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:39:13.276404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:39:13.276404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:39:13.276404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:39:13.276404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:39:13.276404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:13.276404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 547/1089] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:13.281530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23ffbb-0b08-4e85-bb0d-90d8149052be None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:13.325966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ec44-fd9e-4ade-b292-cc506ed100d3 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:13.326253 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ec44-fd9e-4ade-b292-cc506ed100d3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:13.326533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ec44-fd9e-4ade-b292-cc506ed100d3 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:13.327876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 548/1090] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:39:13.333314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74847a82-b776-4418-96c2-77da50b83ea1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:13.333596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74847a82-b776-4418-96c2-77da50b83ea1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:13.333902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74847a82-b776-4418-96c2-77da50b83ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:13.350048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-74847a82-b776-4418-96c2-77da50b83ea1 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:13.351233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-74847a82-b776-4418-96c2-77da50b83ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:13.367254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:13.378244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23ffbb-0b08-4e85-bb0d-90d8149052be None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:13.413348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb421a2-aca9-4ca8-af8b-7580f7f4d94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:13.414051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb421a2-aca9-4ca8-af8b-7580f7f4d94b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:13.414345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb421a2-aca9-4ca8-af8b-7580f7f4d94b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:13.414625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb421a2-aca9-4ca8-af8b-7580f7f4d94b None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:13.415398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb421a2-aca9-4ca8-af8b-7580f7f4d94b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:13.415713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb421a2-aca9-4ca8-af8b-7580f7f4d94b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:13.417144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb421a2-aca9-4ca8-af8b-7580f7f4d94b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:13.417969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 549/1091] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:39:13.433363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 543/1092] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:39:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:13.434551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74847a82-b776-4418-96c2-77da50b83ea1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:13.440424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb421a2-aca9-4ca8-af8b-7580f7f4d94b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:13.466087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:13.532767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74847a82-b776-4418-96c2-77da50b83ea1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:13.539277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb421a2-aca9-4ca8-af8b-7580f7f4d94b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:13.565741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40307b07-60cf-477a-b80e-6b9ff2b0d332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:13.566976 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40307b07-60cf-477a-b80e-6b9ff2b0d332 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:13.567257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40307b07-60cf-477a-b80e-6b9ff2b0d332 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:13.567535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40307b07-60cf-477a-b80e-6b9ff2b0d332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:13.568361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40307b07-60cf-477a-b80e-6b9ff2b0d332 None admin] RBAC: Adding query filter params (role.id=660489b44e624266a7f8293c906652ed, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=dfe8ef12db914a9693ef470f1706bb5a) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:13.568677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40307b07-60cf-477a-b80e-6b9ff2b0d332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:13.570122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40307b07-60cf-477a-b80e-6b9ff2b0d332 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:13.573253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1708a0-6f66-420a-96e0-666e08fd25b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:13.574066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1708a0-6f66-420a-96e0-666e08fd25b7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:13.574353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1708a0-6f66-420a-96e0-666e08fd25b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:13.574626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1708a0-6f66-420a-96e0-666e08fd25b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:13.575926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 550/1093] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:39:13 2022] GET /identity/v3/role_assignments?user.id=dfe8ef12db914a9693ef470f1706bb5a&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=660489b44e624266a7f8293c906652ed => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:13.585666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1708a0-6f66-420a-96e0-666e08fd25b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ec1708a0-6f66-420a-96e0-666e08fd25b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:13.593649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:13.594902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:13.596043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 544/1094] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:13.601316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40307b07-60cf-477a-b80e-6b9ff2b0d332 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:13.621553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:13.708225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40307b07-60cf-477a-b80e-6b9ff2b0d332 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:13.747472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05909d70-85aa-4f25-bf72-12f32bad25c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:13.748159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05909d70-85aa-4f25-bf72-12f32bad25c1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:13.748445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05909d70-85aa-4f25-bf72-12f32bad25c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:13.748721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05909d70-85aa-4f25-bf72-12f32bad25c1 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:13.749480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05909d70-85aa-4f25-bf72-12f32bad25c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:13.749785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05909d70-85aa-4f25-bf72-12f32bad25c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:13.751208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05909d70-85aa-4f25-bf72-12f32bad25c1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:13.760527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 551/1095] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:13.766460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1708a0-6f66-420a-96e0-666e08fd25b7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:13.855063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:13.871337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1708a0-6f66-420a-96e0-666e08fd25b7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:13.904792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1accc-e005-416b-88a7-6454936f71ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:13.905605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1accc-e005-416b-88a7-6454936f71ab None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:13.905944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1accc-e005-416b-88a7-6454936f71ab None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:13.906202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1accc-e005-416b-88a7-6454936f71ab None admin] PATH_INFO: `/v3/users/1b7b3b49edac4c18a7a3eb1f3f9b6304` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:13.907398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1accc-e005-416b-88a7-6454936f71ab None admin] RBAC: Authorizing `identity:get_user(user_id=1b7b3b49edac4c18a7a3eb1f3f9b6304)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:13.908868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1accc-e005-416b-88a7-6454936f71ab None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:13.911052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 545/1096] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:13 2022] GET /identity/v3/users/1b7b3b49edac4c18a7a3eb1f3f9b6304 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:13.917170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05909d70-85aa-4f25-bf72-12f32bad25c1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:13.999942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:14.022721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05909d70-85aa-4f25-bf72-12f32bad25c1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:14.057415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46b123-e2c6-4a79-8ec4-91e42d39aabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:14.058335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46b123-e2c6-4a79-8ec4-91e42d39aabf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:14.058615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46b123-e2c6-4a79-8ec4-91e42d39aabf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:14.058885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46b123-e2c6-4a79-8ec4-91e42d39aabf None admin] PATH_INFO: `/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:14.060041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46b123-e2c6-4a79-8ec4-91e42d39aabf None admin] RBAC: Authorizing `identity:get_project(project_id=eeb90e04b5f2403e9c622416f7e4cefb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:14.061462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46b123-e2c6-4a79-8ec4-91e42d39aabf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:14.063156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 552/1097] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:13 2022] GET /identity/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:14.069986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1accc-e005-416b-88a7-6454936f71ab None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:14.176881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1accc-e005-416b-88a7-6454936f71ab None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:14.211297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619fa7e-296c-4644-9846-3e9afcc26eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:14.212428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619fa7e-296c-4644-9846-3e9afcc26eae None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:14.212708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619fa7e-296c-4644-9846-3e9afcc26eae None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:14.212985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619fa7e-296c-4644-9846-3e9afcc26eae None admin] PATH_INFO: `/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb/users/1b7b3b49edac4c18a7a3eb1f3f9b6304/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:14.215031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619fa7e-296c-4644-9846-3e9afcc26eae None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb90e04b5f2403e9c622416f7e4cefb, user_id=1b7b3b49edac4c18a7a3eb1f3f9b6304, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:14.216482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619fa7e-296c-4644-9846-3e9afcc26eae None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:14.276768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 546/1098] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:39:14 2022] PUT /identity/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb/users/1b7b3b49edac4c18a7a3eb1f3f9b6304/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:39:16.484086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d9c34-a1b3-4fb3-b009-6b6180a3dfbd None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:16.484609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d9c34-a1b3-4fb3-b009-6b6180a3dfbd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:16.484724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d9c34-a1b3-4fb3-b009-6b6180a3dfbd None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:16.486027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 553/1099] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:16.491420 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9c72a-d2db-4570-bb71-8eeed271213e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:16.491827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9c72a-d2db-4570-bb71-8eeed271213e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:16.491977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9c72a-d2db-4570-bb71-8eeed271213e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:16.508803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bfb9c72a-d2db-4570-bb71-8eeed271213e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:16.510011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bfb9c72a-d2db-4570-bb71-8eeed271213e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:16.599157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:16.657190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 547/1100] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:16.681736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d9c34-a1b3-4fb3-b009-6b6180a3dfbd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:16.783058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d9c34-a1b3-4fb3-b009-6b6180a3dfbd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:16.815400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df851050-88c6-49b7-b2b4-587cf174c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:16.816779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df851050-88c6-49b7-b2b4-587cf174c373 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:16.817053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df851050-88c6-49b7-b2b4-587cf174c373 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:16.817321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df851050-88c6-49b7-b2b4-587cf174c373 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:16.826915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df851050-88c6-49b7-b2b4-587cf174c373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-df851050-88c6-49b7-b2b4-587cf174c373 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:16.834512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:16.835671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:16.836722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 554/1101] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:39:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:16.841649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9c72a-d2db-4570-bb71-8eeed271213e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:16.937391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9c72a-d2db-4570-bb71-8eeed271213e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:16.970958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41f191-eaf2-4179-9e3e-92d875fa16b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:16.971663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41f191-eaf2-4179-9e3e-92d875fa16b9 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:16.971951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41f191-eaf2-4179-9e3e-92d875fa16b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:16.972257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41f191-eaf2-4179-9e3e-92d875fa16b9 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:16.972995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41f191-eaf2-4179-9e3e-92d875fa16b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:16.973300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41f191-eaf2-4179-9e3e-92d875fa16b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:16.974701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41f191-eaf2-4179-9e3e-92d875fa16b9 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:16.982550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 548/1102] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:39:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:16.988062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df851050-88c6-49b7-b2b4-587cf174c373 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:16.997122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:17.085197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df851050-88c6-49b7-b2b4-587cf174c373 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:17.117912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf66c4e-c61f-4a8f-bcd2-4a08a37e304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:17.118781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf66c4e-c61f-4a8f-bcd2-4a08a37e304a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:17.119065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf66c4e-c61f-4a8f-bcd2-4a08a37e304a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:17.119344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf66c4e-c61f-4a8f-bcd2-4a08a37e304a None admin] PATH_INFO: `/v3/users/1b7b3b49edac4c18a7a3eb1f3f9b6304` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:17.120513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf66c4e-c61f-4a8f-bcd2-4a08a37e304a None admin] RBAC: Authorizing `identity:get_user(user_id=1b7b3b49edac4c18a7a3eb1f3f9b6304)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:17.121973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf66c4e-c61f-4a8f-bcd2-4a08a37e304a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:17.123598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 555/1103] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:16 2022] GET /identity/v3/users/1b7b3b49edac4c18a7a3eb1f3f9b6304 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:17.129253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41f191-eaf2-4179-9e3e-92d875fa16b9 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:17.133277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:17.224219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41f191-eaf2-4179-9e3e-92d875fa16b9 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:17.255700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e896be-ff9d-4363-853f-9efa5c1428a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:17.256326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e896be-ff9d-4363-853f-9efa5c1428a6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:17.256605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e896be-ff9d-4363-853f-9efa5c1428a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:17.256876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e896be-ff9d-4363-853f-9efa5c1428a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:17.319808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e896be-ff9d-4363-853f-9efa5c1428a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:17.321223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e896be-ff9d-4363-853f-9efa5c1428a6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:17.323692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:17.412912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 549/1104] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:39:17.418948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf66c4e-c61f-4a8f-bcd2-4a08a37e304a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:17.492625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:17.516838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf66c4e-c61f-4a8f-bcd2-4a08a37e304a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:17.550939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415c6b1-47a2-417b-b3f2-e7e80ceb8e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:17.551819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415c6b1-47a2-417b-b3f2-e7e80ceb8e2a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:17.552153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415c6b1-47a2-417b-b3f2-e7e80ceb8e2a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:17.552419 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415c6b1-47a2-417b-b3f2-e7e80ceb8e2a None admin] PATH_INFO: `/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:17.553589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415c6b1-47a2-417b-b3f2-e7e80ceb8e2a None admin] RBAC: Authorizing `identity:get_project(project_id=eeb90e04b5f2403e9c622416f7e4cefb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:17.555053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415c6b1-47a2-417b-b3f2-e7e80ceb8e2a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:17.556713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 556/1105] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:17 2022] GET /identity/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:17.562769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e896be-ff9d-4363-853f-9efa5c1428a6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:17.657030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e896be-ff9d-4363-853f-9efa5c1428a6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:17.659274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:17.689912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a3dc5-37d7-405d-ba7f-555c9643fad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:17.690497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a3dc5-37d7-405d-ba7f-555c9643fad6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:17.691478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a3dc5-37d7-405d-ba7f-555c9643fad6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:17.691802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a3dc5-37d7-405d-ba7f-555c9643fad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:17.692769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a3dc5-37d7-405d-ba7f-555c9643fad6 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=eeb90e04b5f2403e9c622416f7e4cefb, user.id=1b7b3b49edac4c18a7a3eb1f3f9b6304) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:17.693135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a3dc5-37d7-405d-ba7f-555c9643fad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:17.694849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a3dc5-37d7-405d-ba7f-555c9643fad6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:17.700411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 550/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:39:17 2022] GET /identity/v3/role_assignments?user.id=1b7b3b49edac4c18a7a3eb1f3f9b6304&scope.project.id=eeb90e04b5f2403e9c622416f7e4cefb&role.id=fe4d636f5c4842f18569347911e6cdec => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:19.926216 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c74b-f8a0-411e-add9-0444e2f70bf2 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:19.926730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c74b-f8a0-411e-add9-0444e2f70bf2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:19.926837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c74b-f8a0-411e-add9-0444e2f70bf2 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:19.928129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 557/1107] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:19.933793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bb8b4-6777-4fff-bf90-8bf299ce517d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:19.934259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bb8b4-6777-4fff-bf90-8bf299ce517d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:19.934383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bb8b4-6777-4fff-bf90-8bf299ce517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:19.950236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-760bb8b4-6777-4fff-bf90-8bf299ce517d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:19.951412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-760bb8b4-6777-4fff-bf90-8bf299ce517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:20.015652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 551/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:20.040350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c74b-f8a0-411e-add9-0444e2f70bf2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:20.146168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c74b-f8a0-411e-add9-0444e2f70bf2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:20.179563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bd35f-0974-47ad-b725-e4f69a154356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:20.180335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bd35f-0974-47ad-b725-e4f69a154356 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:20.180618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bd35f-0974-47ad-b725-e4f69a154356 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:20.180896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bd35f-0974-47ad-b725-e4f69a154356 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:20.182082 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bd35f-0974-47ad-b725-e4f69a154356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:20.183533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bd35f-0974-47ad-b725-e4f69a154356 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:20.185104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 558/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:20.190753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bb8b4-6777-4fff-bf90-8bf299ce517d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:20.282796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:20.286449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bb8b4-6777-4fff-bf90-8bf299ce517d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:20.319692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f4908-68f2-43c8-9e5f-07a6b021eff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:20.320350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f4908-68f2-43c8-9e5f-07a6b021eff4 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:20.320635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f4908-68f2-43c8-9e5f-07a6b021eff4 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:20.320916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f4908-68f2-43c8-9e5f-07a6b021eff4 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:20.321612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f4908-68f2-43c8-9e5f-07a6b021eff4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:20.323071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f4908-68f2-43c8-9e5f-07a6b021eff4 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:20.405080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 552/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:39:20 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:39:22.638152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8b1b-57b3-42b0-b2a2-b861e7e5a7b6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:22.638680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8b1b-57b3-42b0-b2a2-b861e7e5a7b6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:22.638770 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd8b1b-57b3-42b0-b2a2-b861e7e5a7b6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:22.640057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 559/1111] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:22.645342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2fd2b-041a-4ede-a00b-9058ce0c89cd None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:22.645783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2fd2b-041a-4ede-a00b-9058ce0c89cd None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:22.645906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2fd2b-041a-4ede-a00b-9058ce0c89cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:22.662074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-02a2fd2b-041a-4ede-a00b-9058ce0c89cd None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:22.663500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-02a2fd2b-041a-4ede-a00b-9058ce0c89cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:22.726960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 553/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:22.750838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd8b1b-57b3-42b0-b2a2-b861e7e5a7b6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:22.763960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:22.849363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd8b1b-57b3-42b0-b2a2-b861e7e5a7b6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:22.882163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f471e-58f2-4472-9f74-9942681917f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:22.883001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f471e-58f2-4472-9f74-9942681917f1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:22.883279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f471e-58f2-4472-9f74-9942681917f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:22.883548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f471e-58f2-4472-9f74-9942681917f1 None admin] PATH_INFO: `/v3/roles/7583ad887d8b4052bd11783371c338b4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:22.885021 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f471e-58f2-4472-9f74-9942681917f1 None admin] RBAC: Authorizing `identity:get_role(role_id=7583ad887d8b4052bd11783371c338b4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:22.886423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f471e-58f2-4472-9f74-9942681917f1 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:22.887782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 560/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:22 2022] GET /identity/v3/roles/7583ad887d8b4052bd11783371c338b4 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:22.893544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2fd2b-041a-4ede-a00b-9058ce0c89cd None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:22.974517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:22.990273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2fd2b-041a-4ede-a00b-9058ce0c89cd None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:23.022660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5846d-6811-4fe5-992a-08f028a50db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:23.023525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5846d-6811-4fe5-992a-08f028a50db2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:23.023807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5846d-6811-4fe5-992a-08f028a50db2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:23.024083 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5846d-6811-4fe5-992a-08f028a50db2 None admin] PATH_INFO: `/v3/users/1c014325c5634ac79445fb78caf2fc2f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:23.037206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5846d-6811-4fe5-992a-08f028a50db2 None admin] RBAC: Authorizing `identity:get_user(user_id=1c014325c5634ac79445fb78caf2fc2f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:23.038933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5846d-6811-4fe5-992a-08f028a50db2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:23.040569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 554/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:22 2022] GET /identity/v3/users/1c014325c5634ac79445fb78caf2fc2f => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:23.046236 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f471e-58f2-4472-9f74-9942681917f1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:23.120578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:23.140436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f471e-58f2-4472-9f74-9942681917f1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:23.172950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b70457-4ee3-4072-bd81-74ab9dd15e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:23.173520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b70457-4ee3-4072-bd81-74ab9dd15e8d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:23.173803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b70457-4ee3-4072-bd81-74ab9dd15e8d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:23.174106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b70457-4ee3-4072-bd81-74ab9dd15e8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:23.237928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b70457-4ee3-4072-bd81-74ab9dd15e8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:23.239350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b70457-4ee3-4072-bd81-74ab9dd15e8d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:23.312203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:23.333219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 561/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:39:23.338929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5846d-6811-4fe5-992a-08f028a50db2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:23.434376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5846d-6811-4fe5-992a-08f028a50db2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:23.466977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc286-78cf-432c-95de-a19482684ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:23.467822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc286-78cf-432c-95de-a19482684ce7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:23.468106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc286-78cf-432c-95de-a19482684ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:23.468384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc286-78cf-432c-95de-a19482684ce7 None admin] PATH_INFO: `/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:23.469549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc286-78cf-432c-95de-a19482684ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb90e04b5f2403e9c622416f7e4cefb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:23.471025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc286-78cf-432c-95de-a19482684ce7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:23.473265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 555/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:23 2022] GET /identity/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:23.479097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b70457-4ee3-4072-bd81-74ab9dd15e8d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:23.573266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b70457-4ee3-4072-bd81-74ab9dd15e8d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:23.606472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f8724-163c-4265-9c7f-e75d8ff779a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:23.607104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f8724-163c-4265-9c7f-e75d8ff779a0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:23.607380 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f8724-163c-4265-9c7f-e75d8ff779a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:23.607651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f8724-163c-4265-9c7f-e75d8ff779a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:23.608468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f8724-163c-4265-9c7f-e75d8ff779a0 None admin] RBAC: Adding query filter params (role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=eeb90e04b5f2403e9c622416f7e4cefb, user.id=1c014325c5634ac79445fb78caf2fc2f) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:23.608775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f8724-163c-4265-9c7f-e75d8ff779a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:23.610202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f8724-163c-4265-9c7f-e75d8ff779a0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:23.615460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 562/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:39:23 2022] GET /identity/v3/role_assignments?user.id=1c014325c5634ac79445fb78caf2fc2f&scope.project.id=eeb90e04b5f2403e9c622416f7e4cefb&role.id=7583ad887d8b4052bd11783371c338b4 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:25.822263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1bdcc-05c2-46fc-93cb-72979ca75cdf None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:25.822841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1bdcc-05c2-46fc-93cb-72979ca75cdf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:25.822841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1bdcc-05c2-46fc-93cb-72979ca75cdf None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:25.824185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 556/1118] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:25.829492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf259c-1ae7-4a39-82f7-4a224e6a2a5c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:25.829928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf259c-1ae7-4a39-82f7-4a224e6a2a5c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:25.830081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf259c-1ae7-4a39-82f7-4a224e6a2a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:25.847534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-52bf259c-1ae7-4a39-82f7-4a224e6a2a5c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:25.848690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-52bf259c-1ae7-4a39-82f7-4a224e6a2a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:25.881566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:25.912402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 563/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:25.934830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1bdcc-05c2-46fc-93cb-72979ca75cdf None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:25.947276 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:26.033214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1bdcc-05c2-46fc-93cb-72979ca75cdf None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:26.064967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880bdbd-14a0-4927-9889-06821c48a9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:26.066556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880bdbd-14a0-4927-9889-06821c48a9b1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:26.067160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880bdbd-14a0-4927-9889-06821c48a9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:26.067343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880bdbd-14a0-4927-9889-06821c48a9b1 None admin] PATH_INFO: `/v3/roles/7583ad887d8b4052bd11783371c338b4` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:26.068753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880bdbd-14a0-4927-9889-06821c48a9b1 None admin] RBAC: Authorizing `identity:get_role(role_id=7583ad887d8b4052bd11783371c338b4)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:26.070145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880bdbd-14a0-4927-9889-06821c48a9b1 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:26.071474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 557/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:25 2022] GET /identity/v3/roles/7583ad887d8b4052bd11783371c338b4 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:26.077232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf259c-1ae7-4a39-82f7-4a224e6a2a5c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:26.173526 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf259c-1ae7-4a39-82f7-4a224e6a2a5c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:26.206759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cd9c9-473d-434c-860c-099e03b53288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:26.207740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cd9c9-473d-434c-860c-099e03b53288 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:26.208023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cd9c9-473d-434c-860c-099e03b53288 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:26.208300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cd9c9-473d-434c-860c-099e03b53288 None admin] PATH_INFO: `/v3/users/1c014325c5634ac79445fb78caf2fc2f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:26.209490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cd9c9-473d-434c-860c-099e03b53288 None admin] RBAC: Authorizing `identity:get_user(user_id=1c014325c5634ac79445fb78caf2fc2f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:26.210962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cd9c9-473d-434c-860c-099e03b53288 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:26.212563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 564/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:26 2022] GET /identity/v3/users/1c014325c5634ac79445fb78caf2fc2f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:26.218137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880bdbd-14a0-4927-9889-06821c48a9b1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:26.293191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:26.312979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880bdbd-14a0-4927-9889-06821c48a9b1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:26.345547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bf968-79d8-4d59-98a3-8968cdc41677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:26.346412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bf968-79d8-4d59-98a3-8968cdc41677 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:26.346697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bf968-79d8-4d59-98a3-8968cdc41677 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:26.347025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bf968-79d8-4d59-98a3-8968cdc41677 None admin] PATH_INFO: `/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:26.348178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bf968-79d8-4d59-98a3-8968cdc41677 None admin] RBAC: Authorizing `identity:get_project(project_id=eeb90e04b5f2403e9c622416f7e4cefb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:26.349591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bf968-79d8-4d59-98a3-8968cdc41677 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:26.351266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 558/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:26 2022] GET /identity/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:26.357010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cd9c9-473d-434c-860c-099e03b53288 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:26.398331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:26.454958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cd9c9-473d-434c-860c-099e03b53288 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:26.486973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe885a-d597-4887-ad87-3a4c4e4aec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:26.488555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe885a-d597-4887-ad87-3a4c4e4aec97 None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:26.488836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe885a-d597-4887-ad87-3a4c4e4aec97 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:26.489114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe885a-d597-4887-ad87-3a4c4e4aec97 None admin] PATH_INFO: `/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb/users/1c014325c5634ac79445fb78caf2fc2f/roles/7583ad887d8b4052bd11783371c338b4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:26.491312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe885a-d597-4887-ad87-3a4c4e4aec97 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb90e04b5f2403e9c622416f7e4cefb, user_id=1c014325c5634ac79445fb78caf2fc2f, role_id=7583ad887d8b4052bd11783371c338b4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:26.492732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe885a-d597-4887-ad87-3a4c4e4aec97 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:26.560641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 565/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:39:26 2022] PUT /identity/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb/users/1c014325c5634ac79445fb78caf2fc2f/roles/7583ad887d8b4052bd11783371c338b4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:39:28.755387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762de360-ef69-4adb-9d4f-6b11c7ed7434 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:28.755853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762de360-ef69-4adb-9d4f-6b11c7ed7434 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:28.755944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762de360-ef69-4adb-9d4f-6b11c7ed7434 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:28.757304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 559/1124] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:28.762557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ec61-47e0-40c2-9553-02eab5b0f143 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:28.762972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ec61-47e0-40c2-9553-02eab5b0f143 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:28.763109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ec61-47e0-40c2-9553-02eab5b0f143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:28.779832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5ef5ec61-47e0-40c2-9553-02eab5b0f143 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:28.780997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5ef5ec61-47e0-40c2-9553-02eab5b0f143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:28.844864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 566/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:28.868611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762de360-ef69-4adb-9d4f-6b11c7ed7434 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:28.965676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762de360-ef69-4adb-9d4f-6b11c7ed7434 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:28.997261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2352c-fc6c-4ca0-be80-099cbd55e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:28.998146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2352c-fc6c-4ca0-be80-099cbd55e41c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:28.998436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2352c-fc6c-4ca0-be80-099cbd55e41c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:28.998716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2352c-fc6c-4ca0-be80-099cbd55e41c None admin] PATH_INFO: `/v3/roles/7583ad887d8b4052bd11783371c338b4` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:29.000174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2352c-fc6c-4ca0-be80-099cbd55e41c None admin] RBAC: Authorizing `identity:get_role(role_id=7583ad887d8b4052bd11783371c338b4)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:29.001553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2352c-fc6c-4ca0-be80-099cbd55e41c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:29.003433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 560/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:28 2022] GET /identity/v3/roles/7583ad887d8b4052bd11783371c338b4 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:29.008934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5ec61-47e0-40c2-9553-02eab5b0f143 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:29.038477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:29.104789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5ec61-47e0-40c2-9553-02eab5b0f143 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:29.138048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bad25d-8e97-44ff-9fa3-5265464ebb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:29.138910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bad25d-8e97-44ff-9fa3-5265464ebb00 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:29.139190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bad25d-8e97-44ff-9fa3-5265464ebb00 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:29.139460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bad25d-8e97-44ff-9fa3-5265464ebb00 None admin] PATH_INFO: `/v3/users/1c014325c5634ac79445fb78caf2fc2f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:29.140634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bad25d-8e97-44ff-9fa3-5265464ebb00 None admin] RBAC: Authorizing `identity:get_user(user_id=1c014325c5634ac79445fb78caf2fc2f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:29.142089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bad25d-8e97-44ff-9fa3-5265464ebb00 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:29.144215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 567/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:29 2022] GET /identity/v3/users/1c014325c5634ac79445fb78caf2fc2f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:29.149750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2352c-fc6c-4ca0-be80-099cbd55e41c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:29.165265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:29.244883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2352c-fc6c-4ca0-be80-099cbd55e41c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:29.277298 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49d7b-1043-43b2-8a10-4b8758f84f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:29.277955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49d7b-1043-43b2-8a10-4b8758f84f01 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:29.278241 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49d7b-1043-43b2-8a10-4b8758f84f01 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:29.278517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc49d7b-1043-43b2-8a10-4b8758f84f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:29.342246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc49d7b-1043-43b2-8a10-4b8758f84f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:29.343672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc49d7b-1043-43b2-8a10-4b8758f84f01 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:29.357145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:29.436492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 561/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:39:29.442859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bad25d-8e97-44ff-9fa3-5265464ebb00 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:29.534839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:29.538763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bad25d-8e97-44ff-9fa3-5265464ebb00 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:29.572331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325d1d7-28bb-49ab-a4eb-9893ee1e324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:29.573176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325d1d7-28bb-49ab-a4eb-9893ee1e324c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:29.573459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325d1d7-28bb-49ab-a4eb-9893ee1e324c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:29.573736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325d1d7-28bb-49ab-a4eb-9893ee1e324c None admin] PATH_INFO: `/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:29.574937 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325d1d7-28bb-49ab-a4eb-9893ee1e324c None admin] RBAC: Authorizing `identity:get_project(project_id=eeb90e04b5f2403e9c622416f7e4cefb)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:29.576360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325d1d7-28bb-49ab-a4eb-9893ee1e324c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:29.578038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 568/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:29 2022] GET /identity/v3/projects/eeb90e04b5f2403e9c622416f7e4cefb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:29.584232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49d7b-1043-43b2-8a10-4b8758f84f01 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:29.679513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc49d7b-1043-43b2-8a10-4b8758f84f01 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:29.693388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:29.712300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53400f-debc-41d0-b92b-532ec9d483ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:29.712957 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53400f-debc-41d0-b92b-532ec9d483ce None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:29.713250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53400f-debc-41d0-b92b-532ec9d483ce None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:29.713528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53400f-debc-41d0-b92b-532ec9d483ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:29.714394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53400f-debc-41d0-b92b-532ec9d483ce None admin] RBAC: Adding query filter params (role.id=7583ad887d8b4052bd11783371c338b4, scope.project.id=eeb90e04b5f2403e9c622416f7e4cefb, user.id=1c014325c5634ac79445fb78caf2fc2f) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:29.714706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53400f-debc-41d0-b92b-532ec9d483ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:29.716138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53400f-debc-41d0-b92b-532ec9d483ce None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:29.722932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 562/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:39:29 2022] GET /identity/v3/role_assignments?user.id=1c014325c5634ac79445fb78caf2fc2f&scope.project.id=eeb90e04b5f2403e9c622416f7e4cefb&role.id=7583ad887d8b4052bd11783371c338b4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:31.957613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69839c21-a8e7-4e73-b32e-0c05f9f51b40 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:31.958219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69839c21-a8e7-4e73-b32e-0c05f9f51b40 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:31.958219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69839c21-a8e7-4e73-b32e-0c05f9f51b40 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:31.959533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 569/1131] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:31.964757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999f8e1-082a-4691-a007-c59ce19581e7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:31.965208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999f8e1-082a-4691-a007-c59ce19581e7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:31.965308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999f8e1-082a-4691-a007-c59ce19581e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:31.981438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2999f8e1-082a-4691-a007-c59ce19581e7 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:31.982575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2999f8e1-082a-4691-a007-c59ce19581e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:32.045257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 563/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:32.068720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69839c21-a8e7-4e73-b32e-0c05f9f51b40 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:32.166571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69839c21-a8e7-4e73-b32e-0c05f9f51b40 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:32.199625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23761ec3-c600-40f9-8bce-5c119093e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:32.200395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23761ec3-c600-40f9-8bce-5c119093e8af None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:32.200676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23761ec3-c600-40f9-8bce-5c119093e8af None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:32.200948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23761ec3-c600-40f9-8bce-5c119093e8af None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:32.202131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23761ec3-c600-40f9-8bce-5c119093e8af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:32.203558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23761ec3-c600-40f9-8bce-5c119093e8af None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:32.205125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 570/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:32.210897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999f8e1-082a-4691-a007-c59ce19581e7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:32.304283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999f8e1-082a-4691-a007-c59ce19581e7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:32.312910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:32.336559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0a717-e18c-4c9c-b80c-5cb6e2e06355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:32.337403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a717-e18c-4c9c-b80c-5cb6e2e06355 None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:32.337682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a717-e18c-4c9c-b80c-5cb6e2e06355 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:32.337983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a717-e18c-4c9c-b80c-5cb6e2e06355 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:32.338747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0a717-e18c-4c9c-b80c-5cb6e2e06355 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:32.340164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0a717-e18c-4c9c-b80c-5cb6e2e06355 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:32.422810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 564/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:39:32 2022] POST /identity/v3/projects => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:39:34.667104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42da76-1bf2-4efe-9ec3-00af019f781c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:34.667667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42da76-1bf2-4efe-9ec3-00af019f781c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:34.667667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42da76-1bf2-4efe-9ec3-00af019f781c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:34.668985 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 571/1135] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:34.674327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbca807-060a-4e09-875a-d4fd1edc5798 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:34.675257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbca807-060a-4e09-875a-d4fd1edc5798 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:34.675446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbca807-060a-4e09-875a-d4fd1edc5798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:34.692317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2cbca807-060a-4e09-875a-d4fd1edc5798 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:34.693493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2cbca807-060a-4e09-875a-d4fd1edc5798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:34.757997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 565/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:34.781707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42da76-1bf2-4efe-9ec3-00af019f781c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:34.816231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:34.878672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42da76-1bf2-4efe-9ec3-00af019f781c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:34.912330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92635d-53af-4ee7-91e0-61d11a7cac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:34.913366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92635d-53af-4ee7-91e0-61d11a7cac1b None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:34.913646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92635d-53af-4ee7-91e0-61d11a7cac1b None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:34.913944 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92635d-53af-4ee7-91e0-61d11a7cac1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:34.915111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92635d-53af-4ee7-91e0-61d11a7cac1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:34.916533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92635d-53af-4ee7-91e0-61d11a7cac1b None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:34.918211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 572/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:39:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:34.923950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbca807-060a-4e09-875a-d4fd1edc5798 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:35.011328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:35.020587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbca807-060a-4e09-875a-d4fd1edc5798 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:35.054017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39f179-f035-4316-b573-9df3e4bc795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:35.054775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39f179-f035-4316-b573-9df3e4bc795c None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:35.055059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39f179-f035-4316-b573-9df3e4bc795c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:35.055336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39f179-f035-4316-b573-9df3e4bc795c None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:35.056030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39f179-f035-4316-b573-9df3e4bc795c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:35.057451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39f179-f035-4316-b573-9df3e4bc795c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:35.170289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 566/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:39:34 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:39:37.412174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523cc0a-b1c3-43e1-b043-02e9116278a4 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:37.412954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523cc0a-b1c3-43e1-b043-02e9116278a4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:37.412954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523cc0a-b1c3-43e1-b043-02e9116278a4 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:37.414364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 573/1139] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:37.419669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b8c33-6fde-46e6-b23c-e943f9d3c0d9 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:37.420057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b8c33-6fde-46e6-b23c-e943f9d3c0d9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:37.420336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b8c33-6fde-46e6-b23c-e943f9d3c0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:37.437341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-794b8c33-6fde-46e6-b23c-e943f9d3c0d9 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:37.438552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-794b8c33-6fde-46e6-b23c-e943f9d3c0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:37.503284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 567/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:37.527500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523cc0a-b1c3-43e1-b043-02e9116278a4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:37.625570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523cc0a-b1c3-43e1-b043-02e9116278a4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:37.627078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:37.659470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f82f2-eb1a-4152-96bb-04cdfe855a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:37.660247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f82f2-eb1a-4152-96bb-04cdfe855a00 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:37.660530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f82f2-eb1a-4152-96bb-04cdfe855a00 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:37.660806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f82f2-eb1a-4152-96bb-04cdfe855a00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:37.670900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f82f2-eb1a-4152-96bb-04cdfe855a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-bd1f82f2-eb1a-4152-96bb-04cdfe855a00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:37.678553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:37.684290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:37.685471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 574/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:39:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:37.686415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b8c33-6fde-46e6-b23c-e943f9d3c0d9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:37.759294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:37.780817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b8c33-6fde-46e6-b23c-e943f9d3c0d9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:37.814308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04291979-df9a-47c0-aa75-e2bdb212ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:37.814955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04291979-df9a-47c0-aa75-e2bdb212ba42 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:37.815235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04291979-df9a-47c0-aa75-e2bdb212ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:37.815509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04291979-df9a-47c0-aa75-e2bdb212ba42 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:37.816241 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04291979-df9a-47c0-aa75-e2bdb212ba42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:37.816544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04291979-df9a-47c0-aa75-e2bdb212ba42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:37.817926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04291979-df9a-47c0-aa75-e2bdb212ba42 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:37.826409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 568/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:39:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:37.832704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f82f2-eb1a-4152-96bb-04cdfe855a00 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:37.929052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f82f2-eb1a-4152-96bb-04cdfe855a00 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:37.961190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12259c82-66e4-4b7b-8fa2-43064cb9c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:37.962069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259c82-66e4-4b7b-8fa2-43064cb9c0b9 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:37.962360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259c82-66e4-4b7b-8fa2-43064cb9c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:37.962639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259c82-66e4-4b7b-8fa2-43064cb9c0b9 None admin] PATH_INFO: `/v3/users/32a4a9b3b2b4411e9c04ec1230e818e2` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:37.975819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12259c82-66e4-4b7b-8fa2-43064cb9c0b9 None admin] RBAC: Authorizing `identity:get_user(user_id=32a4a9b3b2b4411e9c04ec1230e818e2)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:37.977244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12259c82-66e4-4b7b-8fa2-43064cb9c0b9 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:37.979070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 575/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:37 2022] GET /identity/v3/users/32a4a9b3b2b4411e9c04ec1230e818e2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:37.984780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04291979-df9a-47c0-aa75-e2bdb212ba42 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:38.080644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04291979-df9a-47c0-aa75-e2bdb212ba42 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:38.113062 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a1d22-d62a-43b7-bf82-af58b16f9463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:38.113688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a1d22-d62a-43b7-bf82-af58b16f9463 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:38.114002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a1d22-d62a-43b7-bf82-af58b16f9463 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:38.114281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a1d22-d62a-43b7-bf82-af58b16f9463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:38.122306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:38.178808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a1d22-d62a-43b7-bf82-af58b16f9463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:38.180295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a1d22-d62a-43b7-bf82-af58b16f9463 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:38.272224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 569/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:39:38.278012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12259c82-66e4-4b7b-8fa2-43064cb9c0b9 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:38.299803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:38.375430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12259c82-66e4-4b7b-8fa2-43064cb9c0b9 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:38.407835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e876a-edfc-4465-a193-ac7ae6431f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:38.408680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e876a-edfc-4465-a193-ac7ae6431f73 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:38.409050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e876a-edfc-4465-a193-ac7ae6431f73 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:38.409327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e876a-edfc-4465-a193-ac7ae6431f73 None admin] PATH_INFO: `/v3/projects/35799de7bcce431fb589865814abb4f8` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:38.410516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e876a-edfc-4465-a193-ac7ae6431f73 None admin] RBAC: Authorizing `identity:get_project(project_id=35799de7bcce431fb589865814abb4f8)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:38.411933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e876a-edfc-4465-a193-ac7ae6431f73 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:38.413849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 576/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:38 2022] GET /identity/v3/projects/35799de7bcce431fb589865814abb4f8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:38.419784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a1d22-d62a-43b7-bf82-af58b16f9463 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:38.458294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:38.515350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a1d22-d62a-43b7-bf82-af58b16f9463 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:38.547340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05c385-dea1-4196-a39c-8746b3211089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:38.547984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05c385-dea1-4196-a39c-8746b3211089 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:38.548268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05c385-dea1-4196-a39c-8746b3211089 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:38.548547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05c385-dea1-4196-a39c-8746b3211089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:38.549951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05c385-dea1-4196-a39c-8746b3211089 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=35799de7bcce431fb589865814abb4f8, user.id=32a4a9b3b2b4411e9c04ec1230e818e2) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:38.550272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05c385-dea1-4196-a39c-8746b3211089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:38.551704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05c385-dea1-4196-a39c-8746b3211089 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:38.556954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 570/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:39:38 2022] GET /identity/v3/role_assignments?user.id=32a4a9b3b2b4411e9c04ec1230e818e2&scope.project.id=35799de7bcce431fb589865814abb4f8&role.id=fe4d636f5c4842f18569347911e6cdec => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:40.720275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f505a-546d-42c1-a1e1-06490d93c912 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:40.720670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f505a-546d-42c1-a1e1-06490d93c912 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:40.720822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f505a-546d-42c1-a1e1-06490d93c912 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:40.722873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 577/1147] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:40.728129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c1885-9a79-4ed5-b59b-671a7c2ade13 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:40.728538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c1885-9a79-4ed5-b59b-671a7c2ade13 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:40.728687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c1885-9a79-4ed5-b59b-671a7c2ade13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:40.744952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-245c1885-9a79-4ed5-b59b-671a7c2ade13 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:40.746163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-245c1885-9a79-4ed5-b59b-671a7c2ade13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:40.809718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 571/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:40.832248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f505a-546d-42c1-a1e1-06490d93c912 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:40.917055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:40.929077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f505a-546d-42c1-a1e1-06490d93c912 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:40.963588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8b48a-2c04-4284-82e0-7fce5add00c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:40.964362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8b48a-2c04-4284-82e0-7fce5add00c6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:40.964641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8b48a-2c04-4284-82e0-7fce5add00c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:40.964916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8b48a-2c04-4284-82e0-7fce5add00c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:40.975218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b8b48a-2c04-4284-82e0-7fce5add00c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-58b8b48a-2c04-4284-82e0-7fce5add00c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:40.982701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:40.983772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:40.984802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 578/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:39:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:40.990403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c1885-9a79-4ed5-b59b-671a7c2ade13 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:41.020144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:41.086249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c1885-9a79-4ed5-b59b-671a7c2ade13 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:41.119470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef2f78-b766-4f4d-8d0c-b97570b01743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:41.120130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef2f78-b766-4f4d-8d0c-b97570b01743 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:41.120416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef2f78-b766-4f4d-8d0c-b97570b01743 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:41.120695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef2f78-b766-4f4d-8d0c-b97570b01743 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:41.121436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef2f78-b766-4f4d-8d0c-b97570b01743 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:41.121744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef2f78-b766-4f4d-8d0c-b97570b01743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:41.123135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef2f78-b766-4f4d-8d0c-b97570b01743 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:41.131081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 572/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:39:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:41.136873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8b48a-2c04-4284-82e0-7fce5add00c6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:41.234517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8b48a-2c04-4284-82e0-7fce5add00c6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:41.266982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63d4f8-cd1b-4258-a564-7932f4364f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:41.267831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d4f8-cd1b-4258-a564-7932f4364f04 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:41.268671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d4f8-cd1b-4258-a564-7932f4364f04 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:41.268949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d4f8-cd1b-4258-a564-7932f4364f04 None admin] PATH_INFO: `/v3/users/32a4a9b3b2b4411e9c04ec1230e818e2` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:41.270149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63d4f8-cd1b-4258-a564-7932f4364f04 None admin] RBAC: Authorizing `identity:get_user(user_id=32a4a9b3b2b4411e9c04ec1230e818e2)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:41.271569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63d4f8-cd1b-4258-a564-7932f4364f04 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:41.273184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 579/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:41 2022] GET /identity/v3/users/32a4a9b3b2b4411e9c04ec1230e818e2 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:41.278756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef2f78-b766-4f4d-8d0c-b97570b01743 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:41.368367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:41.374861 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef2f78-b766-4f4d-8d0c-b97570b01743 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:41.406667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59a560-5dba-45e3-b0fa-f444a2b55a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:41.407704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59a560-5dba-45e3-b0fa-f444a2b55a44 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:41.408032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59a560-5dba-45e3-b0fa-f444a2b55a44 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:41.408643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59a560-5dba-45e3-b0fa-f444a2b55a44 None admin] PATH_INFO: `/v3/projects/35799de7bcce431fb589865814abb4f8` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:41.410455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59a560-5dba-45e3-b0fa-f444a2b55a44 None admin] RBAC: Authorizing `identity:get_project(project_id=35799de7bcce431fb589865814abb4f8)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:41.411940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59a560-5dba-45e3-b0fa-f444a2b55a44 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:41.413893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 573/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:41 2022] GET /identity/v3/projects/35799de7bcce431fb589865814abb4f8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:41.419492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63d4f8-cd1b-4258-a564-7932f4364f04 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:41.437238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:41.514398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63d4f8-cd1b-4258-a564-7932f4364f04 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:41.547152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822fbf0-8001-4a33-a43d-19096f58366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:41.548015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822fbf0-8001-4a33-a43d-19096f58366a None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:41.548291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822fbf0-8001-4a33-a43d-19096f58366a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:41.548563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822fbf0-8001-4a33-a43d-19096f58366a None admin] PATH_INFO: `/v3/projects/35799de7bcce431fb589865814abb4f8/users/32a4a9b3b2b4411e9c04ec1230e818e2/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:41.550494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822fbf0-8001-4a33-a43d-19096f58366a None admin] RBAC: Authorizing `identity:create_grant(project_id=35799de7bcce431fb589865814abb4f8, user_id=32a4a9b3b2b4411e9c04ec1230e818e2, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:41.551885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822fbf0-8001-4a33-a43d-19096f58366a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:41.646589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 580/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:39:41 2022] PUT /identity/v3/projects/35799de7bcce431fb589865814abb4f8/users/32a4a9b3b2b4411e9c04ec1230e818e2/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:39:43.845461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35be49-91bd-4cff-908e-04d4fa00cad1 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:43.845999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35be49-91bd-4cff-908e-04d4fa00cad1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:43.846110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35be49-91bd-4cff-908e-04d4fa00cad1 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:43.847460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 574/1154] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:43.852603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28cffc-60ca-4050-882b-9cacd3c3da2e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:43.853019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28cffc-60ca-4050-882b-9cacd3c3da2e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:43.853171 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28cffc-60ca-4050-882b-9cacd3c3da2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:43.870184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4b28cffc-60ca-4050-882b-9cacd3c3da2e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:43.871386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4b28cffc-60ca-4050-882b-9cacd3c3da2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:43.935992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 581/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:43.958993 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35be49-91bd-4cff-908e-04d4fa00cad1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:44.055811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35be49-91bd-4cff-908e-04d4fa00cad1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:44.086936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddf099-345a-4377-b1ad-308b966b8ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:44.087713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddf099-345a-4377-b1ad-308b966b8ee2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:44.087996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddf099-345a-4377-b1ad-308b966b8ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:44.088269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddf099-345a-4377-b1ad-308b966b8ee2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:44.098895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddf099-345a-4377-b1ad-308b966b8ee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3eddf099-345a-4377-b1ad-308b966b8ee2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:44.106401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:44.107531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:44.108588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 575/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:39:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:44.114053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28cffc-60ca-4050-882b-9cacd3c3da2e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:44.122014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:44.210680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28cffc-60ca-4050-882b-9cacd3c3da2e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:44.243714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20857268-0c80-4fd7-938f-a62bf31cb348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:44.244367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20857268-0c80-4fd7-938f-a62bf31cb348 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:44.244651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20857268-0c80-4fd7-938f-a62bf31cb348 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:44.244930 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20857268-0c80-4fd7-938f-a62bf31cb348 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:44.245667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20857268-0c80-4fd7-938f-a62bf31cb348 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:44.246004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20857268-0c80-4fd7-938f-a62bf31cb348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:44.247361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20857268-0c80-4fd7-938f-a62bf31cb348 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:44.255157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 582/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:39:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:44.260702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddf099-345a-4377-b1ad-308b966b8ee2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:44.284952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:44.355591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddf099-345a-4377-b1ad-308b966b8ee2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:44.387856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b18ab3-dc9b-45c6-811f-2c8ddfea5db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:44.388700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b18ab3-dc9b-45c6-811f-2c8ddfea5db8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:44.388977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b18ab3-dc9b-45c6-811f-2c8ddfea5db8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:44.389247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b18ab3-dc9b-45c6-811f-2c8ddfea5db8 None admin] PATH_INFO: `/v3/users/32a4a9b3b2b4411e9c04ec1230e818e2` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:44.390417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b18ab3-dc9b-45c6-811f-2c8ddfea5db8 None admin] RBAC: Authorizing `identity:get_user(user_id=32a4a9b3b2b4411e9c04ec1230e818e2)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:44.391833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b18ab3-dc9b-45c6-811f-2c8ddfea5db8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:44.393433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 576/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:44 2022] GET /identity/v3/users/32a4a9b3b2b4411e9c04ec1230e818e2 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:44.399012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20857268-0c80-4fd7-938f-a62bf31cb348 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:44.464634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:44.497121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20857268-0c80-4fd7-938f-a62bf31cb348 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:44.530259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05090fec-57d7-4c0d-82b3-4d92392ac2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:44.530897 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05090fec-57d7-4c0d-82b3-4d92392ac2a7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:44.531213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05090fec-57d7-4c0d-82b3-4d92392ac2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:44.531493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05090fec-57d7-4c0d-82b3-4d92392ac2a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:44.596709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05090fec-57d7-4c0d-82b3-4d92392ac2a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:44.598174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05090fec-57d7-4c0d-82b3-4d92392ac2a7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:44.661565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:44.694469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 583/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:39:44.700385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b18ab3-dc9b-45c6-811f-2c8ddfea5db8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:44.789031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:44.798418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b18ab3-dc9b-45c6-811f-2c8ddfea5db8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:44.831942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545f7db-360c-4251-8478-f6bc8cee4376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:44.832794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545f7db-360c-4251-8478-f6bc8cee4376 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:44.833082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545f7db-360c-4251-8478-f6bc8cee4376 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:44.833356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545f7db-360c-4251-8478-f6bc8cee4376 None admin] PATH_INFO: `/v3/projects/35799de7bcce431fb589865814abb4f8` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:44.834622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545f7db-360c-4251-8478-f6bc8cee4376 None admin] RBAC: Authorizing `identity:get_project(project_id=35799de7bcce431fb589865814abb4f8)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:44.836057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545f7db-360c-4251-8478-f6bc8cee4376 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:44.837732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 577/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:44 2022] GET /identity/v3/projects/35799de7bcce431fb589865814abb4f8 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:44.843880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05090fec-57d7-4c0d-82b3-4d92392ac2a7 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:44.941110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05090fec-57d7-4c0d-82b3-4d92392ac2a7 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:44.973613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575618c-920d-4b9b-94af-dc4823521825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:44.974275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575618c-920d-4b9b-94af-dc4823521825 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:44.974558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575618c-920d-4b9b-94af-dc4823521825 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:44.974835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575618c-920d-4b9b-94af-dc4823521825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:44.975658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575618c-920d-4b9b-94af-dc4823521825 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=35799de7bcce431fb589865814abb4f8, user.id=32a4a9b3b2b4411e9c04ec1230e818e2) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:44.975969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575618c-920d-4b9b-94af-dc4823521825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:44.977388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575618c-920d-4b9b-94af-dc4823521825 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:44.984181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 584/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:39:44 2022] GET /identity/v3/role_assignments?user.id=32a4a9b3b2b4411e9c04ec1230e818e2&scope.project.id=35799de7bcce431fb589865814abb4f8&role.id=fe4d636f5c4842f18569347911e6cdec => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:47.200684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069cbfa-82ec-424b-bb10-218916945cd3 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:47.201259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069cbfa-82ec-424b-bb10-218916945cd3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:47.201259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069cbfa-82ec-424b-bb10-218916945cd3 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:47.202647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 578/1162] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:47.207826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e8e4-883a-4085-88c7-b3ab2afa9ece None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:47.208243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e8e4-883a-4085-88c7-b3ab2afa9ece None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:47.208376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e8e4-883a-4085-88c7-b3ab2afa9ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:47.225096 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e341e8e4-883a-4085-88c7-b3ab2afa9ece None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:47.226287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e341e8e4-883a-4085-88c7-b3ab2afa9ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:47.251326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:47.290879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 585/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:47.312984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069cbfa-82ec-424b-bb10-218916945cd3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:47.410361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069cbfa-82ec-424b-bb10-218916945cd3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:47.442555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c18ada-09d9-4f2d-b07f-9efd8684a561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:47.443181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c18ada-09d9-4f2d-b07f-9efd8684a561 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:47.443463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c18ada-09d9-4f2d-b07f-9efd8684a561 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:47.443740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c18ada-09d9-4f2d-b07f-9efd8684a561 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:47.469522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:47.509217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c18ada-09d9-4f2d-b07f-9efd8684a561 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:47.510687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c18ada-09d9-4f2d-b07f-9efd8684a561 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:47.603191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 579/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:39:47.609010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e8e4-883a-4085-88c7-b3ab2afa9ece None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:47.704110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e8e4-883a-4085-88c7-b3ab2afa9ece None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:47.739597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae079e-89a8-44d9-95d7-860cfb7695cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:47.740327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae079e-89a8-44d9-95d7-860cfb7695cf None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:47.740605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae079e-89a8-44d9-95d7-860cfb7695cf None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:47.740878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae079e-89a8-44d9-95d7-860cfb7695cf None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:47.747611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae079e-89a8-44d9-95d7-860cfb7695cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:47.749029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae079e-89a8-44d9-95d7-860cfb7695cf None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-ccae079e-89a8-44d9-95d7-860cfb7695cf None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:47.762609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:47.763890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:39:47.764952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 586/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:39:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:39:47.769814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c18ada-09d9-4f2d-b07f-9efd8684a561 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:47.825305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:47.864906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c18ada-09d9-4f2d-b07f-9efd8684a561 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:47.896569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080f175-d9b1-447b-adf2-a9ccbefbe436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:47.897195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080f175-d9b1-447b-adf2-a9ccbefbe436 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:47.897480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080f175-d9b1-447b-adf2-a9ccbefbe436 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:47.897755 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080f175-d9b1-447b-adf2-a9ccbefbe436 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:47.898499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080f175-d9b1-447b-adf2-a9ccbefbe436 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:47.898802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080f175-d9b1-447b-adf2-a9ccbefbe436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:47.900160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080f175-d9b1-447b-adf2-a9ccbefbe436 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:47.907010 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 580/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Fri Jun 3 09:39:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:50.085115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc03b6-a5da-45bd-87a3-a549e47b064d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:50.085626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc03b6-a5da-45bd-87a3-a549e47b064d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:50.085715 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc03b6-a5da-45bd-87a3-a549e47b064d None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:50.087159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 587/1167] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:50.092410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55923d-4338-4dfb-baa2-a2283cb6780e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:50.092818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55923d-4338-4dfb-baa2-a2283cb6780e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:50.092972 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55923d-4338-4dfb-baa2-a2283cb6780e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:50.109685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2b55923d-4338-4dfb-baa2-a2283cb6780e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:50.110902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2b55923d-4338-4dfb-baa2-a2283cb6780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:50.176020 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 581/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:50.198219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc03b6-a5da-45bd-87a3-a549e47b064d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:50.222814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:50.296898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc03b6-a5da-45bd-87a3-a549e47b064d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:50.329837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62418a-a521-40c9-87d6-be460b5cc3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:50.330557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62418a-a521-40c9-87d6-be460b5cc3f2 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:50.330841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62418a-a521-40c9-87d6-be460b5cc3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:50.331120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62418a-a521-40c9-87d6-be460b5cc3f2 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:50.331844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62418a-a521-40c9-87d6-be460b5cc3f2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:50.333211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62418a-a521-40c9-87d6-be460b5cc3f2 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:50.430555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 588/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Fri Jun 3 09:39:50 2022] POST /identity/v3/domains => generated 267 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:39:52.658583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afde5ae-7b01-4b59-9f1c-6f587895f46f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:52.659149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afde5ae-7b01-4b59-9f1c-6f587895f46f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:52.659149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afde5ae-7b01-4b59-9f1c-6f587895f46f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:52.660492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 582/1170] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:52.666071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb4954-b81d-4334-9483-e059a35b6abb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:52.666509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb4954-b81d-4334-9483-e059a35b6abb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:52.666628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb4954-b81d-4334-9483-e059a35b6abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:52.683684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-8bcb4954-b81d-4334-9483-e059a35b6abb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:52.685146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8bcb4954-b81d-4334-9483-e059a35b6abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:52.753137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 589/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:52.775942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afde5ae-7b01-4b59-9f1c-6f587895f46f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:52.827404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:52.871764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afde5ae-7b01-4b59-9f1c-6f587895f46f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:52.905089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fda4-6872-4b65-ad99-d15554590a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:52.906131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fda4-6872-4b65-ad99-d15554590a27 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:52.906417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fda4-6872-4b65-ad99-d15554590a27 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:52.906689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fda4-6872-4b65-ad99-d15554590a27 None admin] PATH_INFO: `/v3/domains/3daa71d27e7041cc96563c14525272fb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:52.917426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746fda4-6872-4b65-ad99-d15554590a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3daa71d27e7041cc96563c14525272fb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:52.918834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746fda4-6872-4b65-ad99-d15554590a27 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:52.920433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 583/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Fri Jun 3 09:39:52 2022] GET /identity/v3/domains/3daa71d27e7041cc96563c14525272fb => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:52.926861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb4954-b81d-4334-9483-e059a35b6abb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:52.940469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:53.021450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb4954-b81d-4334-9483-e059a35b6abb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:53.055277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f608f-48e9-4751-84e7-c7fc18225541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:53.055919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f608f-48e9-4751-84e7-c7fc18225541 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:53.056206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f608f-48e9-4751-84e7-c7fc18225541 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:53.056484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f608f-48e9-4751-84e7-c7fc18225541 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:53.057182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f608f-48e9-4751-84e7-c7fc18225541 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:53.058656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f608f-48e9-4751-84e7-c7fc18225541 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:53.148484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 590/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:39:52 2022] POST /identity/v3/projects => generated 361 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:39:55.405374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c9aee-e522-4da0-829d-fe68d374e113 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:55.405976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c9aee-e522-4da0-829d-fe68d374e113 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:55.405976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c9aee-e522-4da0-829d-fe68d374e113 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:55.407304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 584/1174] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:55.412820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adec52-0e06-4e72-b790-bd06930c1888 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:55.413242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adec52-0e06-4e72-b790-bd06930c1888 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:55.413382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adec52-0e06-4e72-b790-bd06930c1888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:55.429661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-62adec52-0e06-4e72-b790-bd06930c1888 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:55.430886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-62adec52-0e06-4e72-b790-bd06930c1888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:55.486206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:55.494051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 591/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:55.519418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c9aee-e522-4da0-829d-fe68d374e113 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:55.617063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c9aee-e522-4da0-829d-fe68d374e113 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:55.629201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:55.649317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8639468-2622-4e0a-b436-e4669ace2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:55.650195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8639468-2622-4e0a-b436-e4669ace2fbe None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:55.650483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8639468-2622-4e0a-b436-e4669ace2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:55.650763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8639468-2622-4e0a-b436-e4669ace2fbe None admin] PATH_INFO: `/v3/domains/3daa71d27e7041cc96563c14525272fb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:55.651937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8639468-2622-4e0a-b436-e4669ace2fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=3daa71d27e7041cc96563c14525272fb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:55.653461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8639468-2622-4e0a-b436-e4669ace2fbe None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:55.655097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 585/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Fri Jun 3 09:39:55 2022] GET /identity/v3/domains/3daa71d27e7041cc96563c14525272fb => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:55.660747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adec52-0e06-4e72-b790-bd06930c1888 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:55.756900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adec52-0e06-4e72-b790-bd06930c1888 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:55.790768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47d83f-fa84-40c1-93ae-81b579309d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:55.791430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47d83f-fa84-40c1-93ae-81b579309d60 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:55.791714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47d83f-fa84-40c1-93ae-81b579309d60 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:55.791995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47d83f-fa84-40c1-93ae-81b579309d60 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:55.792700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47d83f-fa84-40c1-93ae-81b579309d60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:55.794159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47d83f-fa84-40c1-93ae-81b579309d60 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:55.884942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 592/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:39:55 2022] POST /identity/v3/users => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:39:58.122766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810682ef-b7cd-4e60-80ee-513915f04609 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:58.123342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810682ef-b7cd-4e60-80ee-513915f04609 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:58.123342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810682ef-b7cd-4e60-80ee-513915f04609 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:58.124673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 586/1178] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:39:58.129985 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb98741-dcdd-464d-ae6c-f6c850903ef4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:58.130367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb98741-dcdd-464d-ae6c-f6c850903ef4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:58.130551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb98741-dcdd-464d-ae6c-f6c850903ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:58.147710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fdb98741-dcdd-464d-ae6c-f6c850903ef4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:39:58.148883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fdb98741-dcdd-464d-ae6c-f6c850903ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:39:58.193996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:58.214058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 593/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:39:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:39:58.237121 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810682ef-b7cd-4e60-80ee-513915f04609 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:58.333821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810682ef-b7cd-4e60-80ee-513915f04609 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:58.365774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165bc14-5a10-4859-8db9-11dd1f234fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:58.366570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165bc14-5a10-4859-8db9-11dd1f234fb1 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:58.366850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165bc14-5a10-4859-8db9-11dd1f234fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:58.367124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165bc14-5a10-4859-8db9-11dd1f234fb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:58.377765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165bc14-5a10-4859-8db9-11dd1f234fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-6165bc14-5a10-4859-8db9-11dd1f234fb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:39:58.385193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:39:58.386339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:39:58.387443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 587/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:39:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:39:58.392361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb98741-dcdd-464d-ae6c-f6c850903ef4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:58.487672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb98741-dcdd-464d-ae6c-f6c850903ef4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:58.520099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3a4b8-3ef9-424b-9a29-3ceeaec23efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:58.520750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3a4b8-3ef9-424b-9a29-3ceeaec23efc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:58.521029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3a4b8-3ef9-424b-9a29-3ceeaec23efc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:58.521302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3a4b8-3ef9-424b-9a29-3ceeaec23efc None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:58.522073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3a4b8-3ef9-424b-9a29-3ceeaec23efc None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:58.522381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3a4b8-3ef9-424b-9a29-3ceeaec23efc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:58.523735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3a4b8-3ef9-424b-9a29-3ceeaec23efc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:58.531455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 594/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:39:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:58.538071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165bc14-5a10-4859-8db9-11dd1f234fb1 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:58.577875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:58.633219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165bc14-5a10-4859-8db9-11dd1f234fb1 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:58.666856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed95e19-fd5b-4063-9358-30f89acc57f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:58.667736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed95e19-fd5b-4063-9358-30f89acc57f8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:58.668023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed95e19-fd5b-4063-9358-30f89acc57f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:58.668301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed95e19-fd5b-4063-9358-30f89acc57f8 None admin] PATH_INFO: `/v3/users/d2a8b7d23ac14392ad6e6f247a98a950` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:58.680833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed95e19-fd5b-4063-9358-30f89acc57f8 None admin] RBAC: Authorizing `identity:get_user(user_id=d2a8b7d23ac14392ad6e6f247a98a950)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:58.682306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed95e19-fd5b-4063-9358-30f89acc57f8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:58.683930 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 588/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:39:58 2022] GET /identity/v3/users/d2a8b7d23ac14392ad6e6f247a98a950 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:58.689536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3a4b8-3ef9-424b-9a29-3ceeaec23efc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:58.732452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:58.787144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3a4b8-3ef9-424b-9a29-3ceeaec23efc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:58.818184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88c145-3b61-4fd3-89c0-63af64057fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:58.818816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c145-3b61-4fd3-89c0-63af64057fb3 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:58.819111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c145-3b61-4fd3-89c0-63af64057fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:58.819387 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c145-3b61-4fd3-89c0-63af64057fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:58.882443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88c145-3b61-4fd3-89c0-63af64057fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:58.883912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88c145-3b61-4fd3-89c0-63af64057fb3 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:58.922790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:58.976289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 595/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:39:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:39:58.982064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed95e19-fd5b-4063-9358-30f89acc57f8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:59.072084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:39:59.075694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed95e19-fd5b-4063-9358-30f89acc57f8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:59.110088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d629b-097b-40df-a1fc-6a2c72f18eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:59.111000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d629b-097b-40df-a1fc-6a2c72f18eac None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:59.111227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d629b-097b-40df-a1fc-6a2c72f18eac None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:59.111508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d629b-097b-40df-a1fc-6a2c72f18eac None admin] PATH_INFO: `/v3/projects/1616c1a338b74df4bb2c3ce6986344f6` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:59.112660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d629b-097b-40df-a1fc-6a2c72f18eac None admin] RBAC: Authorizing `identity:get_project(project_id=1616c1a338b74df4bb2c3ce6986344f6)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:59.114110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d629b-097b-40df-a1fc-6a2c72f18eac None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:59.115773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 589/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:39:58 2022] GET /identity/v3/projects/1616c1a338b74df4bb2c3ce6986344f6 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:39:59.121762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88c145-3b61-4fd3-89c0-63af64057fb3 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:39:59.217117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88c145-3b61-4fd3-89c0-63af64057fb3 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:39:59.256431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b42c71-4656-4787-92c0-9eec8ff1fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:39:59.257094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b42c71-4656-4787-92c0-9eec8ff1fa71 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:39:59.257378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b42c71-4656-4787-92c0-9eec8ff1fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:39:59.257654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b42c71-4656-4787-92c0-9eec8ff1fa71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:39:59.258530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b42c71-4656-4787-92c0-9eec8ff1fa71 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=1616c1a338b74df4bb2c3ce6986344f6, user.id=d2a8b7d23ac14392ad6e6f247a98a950) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:39:59.258846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b42c71-4656-4787-92c0-9eec8ff1fa71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:39:59.260302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b42c71-4656-4787-92c0-9eec8ff1fa71 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:39:59.266149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 596/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:39:59 2022] GET /identity/v3/role_assignments?user.id=d2a8b7d23ac14392ad6e6f247a98a950&scope.project.id=1616c1a338b74df4bb2c3ce6986344f6&role.id=fe4d636f5c4842f18569347911e6cdec => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:40:01.471391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3750b-d3d4-4349-a848-75f83514837f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:01.478251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3750b-d3d4-4349-a848-75f83514837f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:01.478251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3750b-d3d4-4349-a848-75f83514837f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:01.478251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 590/1186] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:40:01.478708 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55f6e3-2411-424f-aa78-21e39fee592a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:01.478708 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55f6e3-2411-424f-aa78-21e39fee592a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:01.479164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55f6e3-2411-424f-aa78-21e39fee592a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:01.490163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:01.495849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-dd55f6e3-2411-424f-aa78-21e39fee592a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:40:01.497151 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dd55f6e3-2411-424f-aa78-21e39fee592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:40:01.561630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 597/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:40:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:40:01.584226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3750b-d3d4-4349-a848-75f83514837f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:01.680334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3750b-d3d4-4349-a848-75f83514837f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:01.712142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa9524-b1c9-46fe-b232-e267dff00341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:01.712923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa9524-b1c9-46fe-b232-e267dff00341 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:01.713203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa9524-b1c9-46fe-b232-e267dff00341 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:01.713479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa9524-b1c9-46fe-b232-e267dff00341 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:01.724397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa9524-b1c9-46fe-b232-e267dff00341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-ffaa9524-b1c9-46fe-b232-e267dff00341 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:40:01.732060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:40:01.733116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:40:01.734046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 591/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:40:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:40:01.739122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55f6e3-2411-424f-aa78-21e39fee592a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:01.833717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55f6e3-2411-424f-aa78-21e39fee592a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:01.865902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36928e04-7849-4545-b052-fdd4afaf6c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:01.866571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36928e04-7849-4545-b052-fdd4afaf6c50 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:01.866854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36928e04-7849-4545-b052-fdd4afaf6c50 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:01.867126 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36928e04-7849-4545-b052-fdd4afaf6c50 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:01.867859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36928e04-7849-4545-b052-fdd4afaf6c50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:40:01.868162 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36928e04-7849-4545-b052-fdd4afaf6c50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:40:01.869538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36928e04-7849-4545-b052-fdd4afaf6c50 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:40:01.871646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:01.877626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 598/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:40:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:40:01.883375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa9524-b1c9-46fe-b232-e267dff00341 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:01.911240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:01.981181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa9524-b1c9-46fe-b232-e267dff00341 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:02.013274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05e9f8-06ef-4e2e-be02-224ccf0d1720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:02.014162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05e9f8-06ef-4e2e-be02-224ccf0d1720 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:02.014452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05e9f8-06ef-4e2e-be02-224ccf0d1720 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:02.014733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05e9f8-06ef-4e2e-be02-224ccf0d1720 None admin] PATH_INFO: `/v3/users/d2a8b7d23ac14392ad6e6f247a98a950` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:02.015906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05e9f8-06ef-4e2e-be02-224ccf0d1720 None admin] RBAC: Authorizing `identity:get_user(user_id=d2a8b7d23ac14392ad6e6f247a98a950)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:40:02.017989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05e9f8-06ef-4e2e-be02-224ccf0d1720 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:40:02.019590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 592/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:40:01 2022] GET /identity/v3/users/d2a8b7d23ac14392ad6e6f247a98a950 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:40:02.025151 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36928e04-7849-4545-b052-fdd4afaf6c50 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:02.121164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36928e04-7849-4545-b052-fdd4afaf6c50 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:02.152611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f92919-fad0-43d0-8492-f1089fb24af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:02.153439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f92919-fad0-43d0-8492-f1089fb24af7 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:02.153720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f92919-fad0-43d0-8492-f1089fb24af7 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:02.154025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f92919-fad0-43d0-8492-f1089fb24af7 None admin] PATH_INFO: `/v3/projects/1616c1a338b74df4bb2c3ce6986344f6` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:02.155177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f92919-fad0-43d0-8492-f1089fb24af7 None admin] RBAC: Authorizing `identity:get_project(project_id=1616c1a338b74df4bb2c3ce6986344f6)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:40:02.156607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f92919-fad0-43d0-8492-f1089fb24af7 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:40:02.158807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 599/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:40:02 2022] GET /identity/v3/projects/1616c1a338b74df4bb2c3ce6986344f6 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:40:02.164867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05e9f8-06ef-4e2e-be02-224ccf0d1720 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:02.254285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:02.260454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05e9f8-06ef-4e2e-be02-224ccf0d1720 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:02.293188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d5308-e401-4a92-8a75-2561f2b9ad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:02.294079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d5308-e401-4a92-8a75-2561f2b9ad8c None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:02.294366 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d5308-e401-4a92-8a75-2561f2b9ad8c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:02.294637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d5308-e401-4a92-8a75-2561f2b9ad8c None admin] PATH_INFO: `/v3/projects/1616c1a338b74df4bb2c3ce6986344f6/users/d2a8b7d23ac14392ad6e6f247a98a950/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:02.296528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d5308-e401-4a92-8a75-2561f2b9ad8c None admin] RBAC: Authorizing `identity:create_grant(project_id=1616c1a338b74df4bb2c3ce6986344f6, user_id=d2a8b7d23ac14392ad6e6f247a98a950, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:40:02.297951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d5308-e401-4a92-8a75-2561f2b9ad8c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:40:02.369237 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 593/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:40:02 2022] PUT /identity/v3/projects/1616c1a338b74df4bb2c3ce6986344f6/users/d2a8b7d23ac14392ad6e6f247a98a950/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:40:04.552265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdaf55d-4d08-45c2-b108-429ce160b65b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:04.552792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdaf55d-4d08-45c2-b108-429ce160b65b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:04.552792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdaf55d-4d08-45c2-b108-429ce160b65b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:04.554194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 600/1193] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:40:04.559416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adff4b-a2d0-430f-aeb6-22ac62de261f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:04.559838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adff4b-a2d0-430f-aeb6-22ac62de261f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:04.559980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adff4b-a2d0-430f-aeb6-22ac62de261f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:04.577521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-67adff4b-a2d0-430f-aeb6-22ac62de261f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:40:04.578744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-67adff4b-a2d0-430f-aeb6-22ac62de261f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:40:04.641708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 594/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:40:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:40:04.664600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdaf55d-4d08-45c2-b108-429ce160b65b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:04.731958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:04.766726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdaf55d-4d08-45c2-b108-429ce160b65b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:04.799063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16849890-52a3-46a0-bdaa-c852484503ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:04.799838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16849890-52a3-46a0-bdaa-c852484503ce None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:04.800120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16849890-52a3-46a0-bdaa-c852484503ce None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:04.800396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16849890-52a3-46a0-bdaa-c852484503ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:04.810120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16849890-52a3-46a0-bdaa-c852484503ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-16849890-52a3-46a0-bdaa-c852484503ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:40:04.818246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:40:04.819343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:40:04.820445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 601/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:40:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:40:04.825347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adff4b-a2d0-430f-aeb6-22ac62de261f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:04.903124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:04.918521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adff4b-a2d0-430f-aeb6-22ac62de261f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:04.950617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4f667-00dc-4fd7-adf8-efec60f3615c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:04.951268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4f667-00dc-4fd7-adf8-efec60f3615c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:04.951547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4f667-00dc-4fd7-adf8-efec60f3615c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:04.951820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4f667-00dc-4fd7-adf8-efec60f3615c None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:04.952547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4f667-00dc-4fd7-adf8-efec60f3615c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:40:04.952846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4f667-00dc-4fd7-adf8-efec60f3615c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:40:04.954275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4f667-00dc-4fd7-adf8-efec60f3615c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:40:04.961847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 595/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:40:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:40:04.967392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16849890-52a3-46a0-bdaa-c852484503ce None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:05.064547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16849890-52a3-46a0-bdaa-c852484503ce None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:05.087898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:05.096805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22dc42-4785-42f6-a4ab-a8aa12a5c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:05.100160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22dc42-4785-42f6-a4ab-a8aa12a5c88d None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:05.100160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22dc42-4785-42f6-a4ab-a8aa12a5c88d None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:05.100160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22dc42-4785-42f6-a4ab-a8aa12a5c88d None admin] PATH_INFO: `/v3/users/d2a8b7d23ac14392ad6e6f247a98a950` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:05.100160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22dc42-4785-42f6-a4ab-a8aa12a5c88d None admin] RBAC: Authorizing `identity:get_user(user_id=d2a8b7d23ac14392ad6e6f247a98a950)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:40:05.100830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22dc42-4785-42f6-a4ab-a8aa12a5c88d None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:40:05.102471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 602/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Fri Jun 3 09:40:04 2022] GET /identity/v3/users/d2a8b7d23ac14392ad6e6f247a98a950 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:40:05.108294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4f667-00dc-4fd7-adf8-efec60f3615c None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:05.205103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4f667-00dc-4fd7-adf8-efec60f3615c None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:05.237624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90debe1-4b8d-4d9a-975f-1c36184aa03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:05.238290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90debe1-4b8d-4d9a-975f-1c36184aa03e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:05.238575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90debe1-4b8d-4d9a-975f-1c36184aa03e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:05.238852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90debe1-4b8d-4d9a-975f-1c36184aa03e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:05.253147 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:05.303665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90debe1-4b8d-4d9a-975f-1c36184aa03e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:40:05.305107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90debe1-4b8d-4d9a-975f-1c36184aa03e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:40:05.399244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 596/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:40:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:40:05.405020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22dc42-4785-42f6-a4ab-a8aa12a5c88d None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:05.501641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22dc42-4785-42f6-a4ab-a8aa12a5c88d None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:05.533746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3030822-1da0-4a4c-a7f0-2b761b25f0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:05.534605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3030822-1da0-4a4c-a7f0-2b761b25f0e6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:05.534887 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3030822-1da0-4a4c-a7f0-2b761b25f0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:05.535161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3030822-1da0-4a4c-a7f0-2b761b25f0e6 None admin] PATH_INFO: `/v3/projects/1616c1a338b74df4bb2c3ce6986344f6` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:05.536311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3030822-1da0-4a4c-a7f0-2b761b25f0e6 None admin] RBAC: Authorizing `identity:get_project(project_id=1616c1a338b74df4bb2c3ce6986344f6)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:40:05.538027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3030822-1da0-4a4c-a7f0-2b761b25f0e6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:40:05.539663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 603/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Fri Jun 3 09:40:05 2022] GET /identity/v3/projects/1616c1a338b74df4bb2c3ce6986344f6 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:40:05.545408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90debe1-4b8d-4d9a-975f-1c36184aa03e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:40:05.589411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:05.639538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90debe1-4b8d-4d9a-975f-1c36184aa03e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:40:05.671827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a580878-9d9d-4b97-81c3-20de41f4ad10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:40:05.672411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a580878-9d9d-4b97-81c3-20de41f4ad10 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:40:05.672693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a580878-9d9d-4b97-81c3-20de41f4ad10 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:40:05.672968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a580878-9d9d-4b97-81c3-20de41f4ad10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:40:05.673811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a580878-9d9d-4b97-81c3-20de41f4ad10 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=1616c1a338b74df4bb2c3ce6986344f6, user.id=d2a8b7d23ac14392ad6e6f247a98a950) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:40:05.674153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a580878-9d9d-4b97-81c3-20de41f4ad10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:40:05.675565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a580878-9d9d-4b97-81c3-20de41f4ad10 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:40:05.681120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 597/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:40:05 2022] GET /identity/v3/role_assignments?user.id=d2a8b7d23ac14392ad6e6f247a98a950&scope.project.id=1616c1a338b74df4bb2c3ce6986344f6&role.id=fe4d636f5c4842f18569347911e6cdec => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:40:15.529716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=69 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:40:15.678568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=45 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:26.498656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac371a52-37d8-4e2c-9c2c-f79d48f7741c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:26.499229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac371a52-37d8-4e2c-9c2c-f79d48f7741c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:26.499229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac371a52-37d8-4e2c-9c2c-f79d48f7741c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:26.500576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 604/1201] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:41:26.506296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183ace1-b3a9-4383-8660-9ca27ab62911 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:26.506774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183ace1-b3a9-4383-8660-9ca27ab62911 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:26.506904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183ace1-b3a9-4383-8660-9ca27ab62911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:26.523662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0183ace1-b3a9-4383-8660-9ca27ab62911 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:41:26.524818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0183ace1-b3a9-4383-8660-9ca27ab62911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:41:26.588631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 598/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:41:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:41:26.613862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac371a52-37d8-4e2c-9c2c-f79d48f7741c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:26.712784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac371a52-37d8-4e2c-9c2c-f79d48f7741c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:26.746179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049238fc-657f-4948-8ec0-2ca407e311b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:26.747034 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049238fc-657f-4948-8ec0-2ca407e311b5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:26.747240 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049238fc-657f-4948-8ec0-2ca407e311b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:26.747516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049238fc-657f-4948-8ec0-2ca407e311b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:26.757855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049238fc-657f-4948-8ec0-2ca407e311b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:26.759277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049238fc-657f-4948-8ec0-2ca407e311b5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-049238fc-657f-4948-8ec0-2ca407e311b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:26.776215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:26.777317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:26.778550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:26.778550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:26.778550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:26.778550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:26.778550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 605/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:26.783554 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183ace1-b3a9-4383-8660-9ca27ab62911 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:26.880704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183ace1-b3a9-4383-8660-9ca27ab62911 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:26.907967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:26.914166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ccfa8-c40b-4ad9-8221-41435b16869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:26.914814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ccfa8-c40b-4ad9-8221-41435b16869c None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:26.915146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ccfa8-c40b-4ad9-8221-41435b16869c None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:26.915423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ccfa8-c40b-4ad9-8221-41435b16869c None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:26.916151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ccfa8-c40b-4ad9-8221-41435b16869c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:26.916459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ccfa8-c40b-4ad9-8221-41435b16869c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:26.917901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ccfa8-c40b-4ad9-8221-41435b16869c None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:26.929901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 599/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:26.935630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049238fc-657f-4948-8ec0-2ca407e311b5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:26.988028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:27.031732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049238fc-657f-4948-8ec0-2ca407e311b5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:27.065769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419866c-2f31-4f6e-a65e-8df170aed982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:27.066447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419866c-2f31-4f6e-a65e-8df170aed982 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:27.066726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419866c-2f31-4f6e-a65e-8df170aed982 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:27.066999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419866c-2f31-4f6e-a65e-8df170aed982 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:27.067696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419866c-2f31-4f6e-a65e-8df170aed982 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:27.069110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419866c-2f31-4f6e-a65e-8df170aed982 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:27.159364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 606/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Fri Jun 3 09:41:26 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:41:29.413275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6898f4-919c-4b78-83ac-97cac31df967 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:29.413925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6898f4-919c-4b78-83ac-97cac31df967 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:29.413925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6898f4-919c-4b78-83ac-97cac31df967 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:29.415279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 600/1206] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:41:29.420411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bd938-b888-4fbd-ba55-de0787b1f735 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:29.420809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bd938-b888-4fbd-ba55-de0787b1f735 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:29.420969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bd938-b888-4fbd-ba55-de0787b1f735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:29.438359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ce4bd938-b888-4fbd-ba55-de0787b1f735 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:41:29.442490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ce4bd938-b888-4fbd-ba55-de0787b1f735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:41:29.505395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 607/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:41:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:41:29.529804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6898f4-919c-4b78-83ac-97cac31df967 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:29.629133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6898f4-919c-4b78-83ac-97cac31df967 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:29.661216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef66726-255d-4e0d-b6b5-5a77c2a28153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:29.662037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef66726-255d-4e0d-b6b5-5a77c2a28153 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:29.662326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef66726-255d-4e0d-b6b5-5a77c2a28153 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:29.662607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef66726-255d-4e0d-b6b5-5a77c2a28153 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:29.678250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef66726-255d-4e0d-b6b5-5a77c2a28153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8ef66726-255d-4e0d-b6b5-5a77c2a28153 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:41:29.681041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:29.682229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:29.683156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 601/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:41:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:29.688232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bd938-b888-4fbd-ba55-de0787b1f735 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:29.727048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:29.783835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bd938-b888-4fbd-ba55-de0787b1f735 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:29.818164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f36f7-3663-41a5-a4bd-f1b456fe6786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:29.818845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f36f7-3663-41a5-a4bd-f1b456fe6786 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:29.819127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f36f7-3663-41a5-a4bd-f1b456fe6786 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:29.819404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f36f7-3663-41a5-a4bd-f1b456fe6786 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:29.820140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f36f7-3663-41a5-a4bd-f1b456fe6786 None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:29.820440 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f36f7-3663-41a5-a4bd-f1b456fe6786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:29.821799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f36f7-3663-41a5-a4bd-f1b456fe6786 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:29.830067 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 608/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:41:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:29.835305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef66726-255d-4e0d-b6b5-5a77c2a28153 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:29.885024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:29.939679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef66726-255d-4e0d-b6b5-5a77c2a28153 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:29.964853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f0495-5cb7-4fb4-85ba-fc3cc47f6720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:29.965641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f0495-5cb7-4fb4-85ba-fc3cc47f6720 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:29.965948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f0495-5cb7-4fb4-85ba-fc3cc47f6720 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:29.966233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f0495-5cb7-4fb4-85ba-fc3cc47f6720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:29.976691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f0495-5cb7-4fb4-85ba-fc3cc47f6720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:29.978174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f0495-5cb7-4fb4-85ba-fc3cc47f6720 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-9d8f0495-5cb7-4fb4-85ba-fc3cc47f6720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:29.995342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:29.996383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:29.997564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:29.997564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:29.997564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:29.997564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:29.997564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 602/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:30.002769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f36f7-3663-41a5-a4bd-f1b456fe6786 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:30.097354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:30.100139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f36f7-3663-41a5-a4bd-f1b456fe6786 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:30.132381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c352d3-1fdf-4df0-ab92-cf76ee9271de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:30.133042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c352d3-1fdf-4df0-ab92-cf76ee9271de None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:30.133330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c352d3-1fdf-4df0-ab92-cf76ee9271de None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:30.133611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c352d3-1fdf-4df0-ab92-cf76ee9271de None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:30.134364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c352d3-1fdf-4df0-ab92-cf76ee9271de None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:30.134675 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c352d3-1fdf-4df0-ab92-cf76ee9271de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:30.136038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c352d3-1fdf-4df0-ab92-cf76ee9271de None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:30.147593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 609/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:30.153349 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f0495-5cb7-4fb4-85ba-fc3cc47f6720 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:30.248459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f0495-5cb7-4fb4-85ba-fc3cc47f6720 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:30.249858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:30.281272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c763e-353d-4116-87d7-fd7a7ef4a86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:30.282108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c763e-353d-4116-87d7-fd7a7ef4a86f None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:30.282421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c763e-353d-4116-87d7-fd7a7ef4a86f None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:30.282695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c763e-353d-4116-87d7-fd7a7ef4a86f None admin] PATH_INFO: `/v3/users/placement` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:30.289521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c763e-353d-4116-87d7-fd7a7ef4a86f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:30.290969 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c763e-353d-4116-87d7-fd7a7ef4a86f None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-277c763e-353d-4116-87d7-fd7a7ef4a86f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:41:30.305042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:30.306110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:30.307179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 603/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:30.312207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c352d3-1fdf-4df0-ab92-cf76ee9271de None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:30.409117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c352d3-1fdf-4df0-ab92-cf76ee9271de None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:30.441932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49684e8-f5ce-4133-a55f-eeb1c4764970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:30.442614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49684e8-f5ce-4133-a55f-eeb1c4764970 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:30.442893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49684e8-f5ce-4133-a55f-eeb1c4764970 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:30.443169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49684e8-f5ce-4133-a55f-eeb1c4764970 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:30.443986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49684e8-f5ce-4133-a55f-eeb1c4764970 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:30.444291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49684e8-f5ce-4133-a55f-eeb1c4764970 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:30.445695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49684e8-f5ce-4133-a55f-eeb1c4764970 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:30.462129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 610/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Fri Jun 3 09:41:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:30.467806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c763e-353d-4116-87d7-fd7a7ef4a86f None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:30.564605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c763e-353d-4116-87d7-fd7a7ef4a86f None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:30.597160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae98b4c-cf33-4e31-8641-af0e4b6a2e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:30.597792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae98b4c-cf33-4e31-8641-af0e4b6a2e53 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:30.598105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae98b4c-cf33-4e31-8641-af0e4b6a2e53 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:30.598385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae98b4c-cf33-4e31-8641-af0e4b6a2e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:30.627409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:30.662514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae98b4c-cf33-4e31-8641-af0e4b6a2e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:30.663948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae98b4c-cf33-4e31-8641-af0e4b6a2e53 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:30.759070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 604/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:41:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:41:30.764992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49684e8-f5ce-4133-a55f-eeb1c4764970 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:30.783811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:30.861859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49684e8-f5ce-4133-a55f-eeb1c4764970 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:30.894694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f776128-b153-4025-b2ec-e1625cd29ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:30.895488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f776128-b153-4025-b2ec-e1625cd29ad5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:30.895774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f776128-b153-4025-b2ec-e1625cd29ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:30.896095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f776128-b153-4025-b2ec-e1625cd29ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:30.906132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f776128-b153-4025-b2ec-e1625cd29ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:30.908115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f776128-b153-4025-b2ec-e1625cd29ad5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-9f776128-b153-4025-b2ec-e1625cd29ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:30.926868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:30.927929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:30.933510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:30.933510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:30.933510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:30.933510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:30.933510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 611/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:30.933876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae98b4c-cf33-4e31-8641-af0e4b6a2e53 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:30.993302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:31.030147 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae98b4c-cf33-4e31-8641-af0e4b6a2e53 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:31.064509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90a35a-750b-4048-972a-221b0d9cd0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:31.065164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90a35a-750b-4048-972a-221b0d9cd0fe None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:31.065449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90a35a-750b-4048-972a-221b0d9cd0fe None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:31.065729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90a35a-750b-4048-972a-221b0d9cd0fe None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:31.066500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90a35a-750b-4048-972a-221b0d9cd0fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:31.066839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90a35a-750b-4048-972a-221b0d9cd0fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:31.068199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90a35a-750b-4048-972a-221b0d9cd0fe None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:31.079266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 605/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:31.084873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f776128-b153-4025-b2ec-e1625cd29ad5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:31.153816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:31.184039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f776128-b153-4025-b2ec-e1625cd29ad5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:31.217015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837fde8-676c-494d-838a-498adbc2d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:31.220953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837fde8-676c-494d-838a-498adbc2d716 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:31.220953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837fde8-676c-494d-838a-498adbc2d716 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:31.220953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837fde8-676c-494d-838a-498adbc2d716 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:31.225517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837fde8-676c-494d-838a-498adbc2d716 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:31.227054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837fde8-676c-494d-838a-498adbc2d716 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6837fde8-676c-494d-838a-498adbc2d716 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:41:31.240746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:31.246091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:31.247202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:41:31.247202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:41:31.247202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:41:31.247202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:41:31.247202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:41:31.247202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:41:31.247202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:31.247202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:31.247202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 612/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:41:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:41:31.247837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90a35a-750b-4048-972a-221b0d9cd0fe None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:31.346631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90a35a-750b-4048-972a-221b0d9cd0fe None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:31.369037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:31.380558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78551771-491b-4f47-af76-6b5a21b60e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:31.381219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78551771-491b-4f47-af76-6b5a21b60e1b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:31.381503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78551771-491b-4f47-af76-6b5a21b60e1b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:31.381776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78551771-491b-4f47-af76-6b5a21b60e1b None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:31.382608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78551771-491b-4f47-af76-6b5a21b60e1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:31.382915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78551771-491b-4f47-af76-6b5a21b60e1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:31.384368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78551771-491b-4f47-af76-6b5a21b60e1b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:31.396362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 606/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:41:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:31.402489 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837fde8-676c-494d-838a-498adbc2d716 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:31.503951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837fde8-676c-494d-838a-498adbc2d716 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:31.532059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:31.538399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0d1e3-e372-412a-b6f2-2a66de2e0d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:31.546057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0d1e3-e372-412a-b6f2-2a66de2e0d32 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:31.546057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0d1e3-e372-412a-b6f2-2a66de2e0d32 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:31.546057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0d1e3-e372-412a-b6f2-2a66de2e0d32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:31.546057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0d1e3-e372-412a-b6f2-2a66de2e0d32 None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=566c0c4bdcaf4c13b060af4bbff5bc04) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:31.546057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0d1e3-e372-412a-b6f2-2a66de2e0d32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:31.546057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0d1e3-e372-412a-b6f2-2a66de2e0d32 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:31.548101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 613/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:41:31 2022] GET /identity/v3/role_assignments?user.id=566c0c4bdcaf4c13b060af4bbff5bc04&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:33.740091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda1496-0bcf-4df3-895e-0f607172141f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:33.740570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda1496-0bcf-4df3-895e-0f607172141f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:33.740662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda1496-0bcf-4df3-895e-0f607172141f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:33.742043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 607/1220] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:41:33.747337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dda4c-4468-46f3-875c-1c87675cbe66 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:33.747761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dda4c-4468-46f3-875c-1c87675cbe66 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:33.747905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3dda4c-4468-46f3-875c-1c87675cbe66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:33.765078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-8d3dda4c-4468-46f3-875c-1c87675cbe66 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:41:33.767499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8d3dda4c-4468-46f3-875c-1c87675cbe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:41:33.831877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 614/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:41:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:41:33.855627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda1496-0bcf-4df3-895e-0f607172141f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:33.953548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda1496-0bcf-4df3-895e-0f607172141f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:33.986323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8870c-14bb-4e8e-b858-02a94ba43bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:33.987047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8870c-14bb-4e8e-b858-02a94ba43bd2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:33.987330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8870c-14bb-4e8e-b858-02a94ba43bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:33.987605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8870c-14bb-4e8e-b858-02a94ba43bd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:33.997324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8870c-14bb-4e8e-b858-02a94ba43bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-37c8870c-14bb-4e8e-b858-02a94ba43bd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:41:34.005462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:34.006643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:34.007684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 608/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:41:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:34.012713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dda4c-4468-46f3-875c-1c87675cbe66 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:34.109183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3dda4c-4468-46f3-875c-1c87675cbe66 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:34.133940 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:34.144200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b1908-9111-4629-a8e7-0167fea22efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:34.144857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b1908-9111-4629-a8e7-0167fea22efe None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:34.145143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b1908-9111-4629-a8e7-0167fea22efe None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:34.145421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b1908-9111-4629-a8e7-0167fea22efe None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:34.146205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b1908-9111-4629-a8e7-0167fea22efe None admin] RBAC: Adding query filter params (name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:34.146514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b1908-9111-4629-a8e7-0167fea22efe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:34.147910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b1908-9111-4629-a8e7-0167fea22efe None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:34.155711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 615/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:41:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:34.161778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8870c-14bb-4e8e-b858-02a94ba43bd2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:34.205410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:34.258944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8870c-14bb-4e8e-b858-02a94ba43bd2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:34.290703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497909a0-25db-4324-adb5-6164eefc4402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:34.296334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497909a0-25db-4324-adb5-6164eefc4402 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:34.296334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497909a0-25db-4324-adb5-6164eefc4402 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:34.296334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497909a0-25db-4324-adb5-6164eefc4402 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:34.303110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497909a0-25db-4324-adb5-6164eefc4402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:34.304593 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497909a0-25db-4324-adb5-6164eefc4402 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-497909a0-25db-4324-adb5-6164eefc4402 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:34.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:34.323003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:34.324112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:34.324112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:34.324112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:34.324112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:34.324112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 609/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:34.328981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b1908-9111-4629-a8e7-0167fea22efe None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:34.428583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b1908-9111-4629-a8e7-0167fea22efe None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:34.460996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239793e-9b92-4cfc-8427-6863d17af3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:34.461641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239793e-9b92-4cfc-8427-6863d17af3fc None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:34.461941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239793e-9b92-4cfc-8427-6863d17af3fc None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:34.462216 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239793e-9b92-4cfc-8427-6863d17af3fc None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:34.462927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239793e-9b92-4cfc-8427-6863d17af3fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:34.463230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239793e-9b92-4cfc-8427-6863d17af3fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:34.464584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239793e-9b92-4cfc-8427-6863d17af3fc None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:34.476398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 616/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:34.482352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497909a0-25db-4324-adb5-6164eefc4402 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:34.574618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:34.581118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497909a0-25db-4324-adb5-6164eefc4402 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:34.614188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701e7ba-5475-4e20-ac66-9bdd76d55b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:34.614981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701e7ba-5475-4e20-ac66-9bdd76d55b47 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:34.615265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701e7ba-5475-4e20-ac66-9bdd76d55b47 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:34.615543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701e7ba-5475-4e20-ac66-9bdd76d55b47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:34.622684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701e7ba-5475-4e20-ac66-9bdd76d55b47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:34.624203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701e7ba-5475-4e20-ac66-9bdd76d55b47 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3701e7ba-5475-4e20-ac66-9bdd76d55b47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:41:34.638342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:34.639427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:34.640568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 610/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:34.645517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239793e-9b92-4cfc-8427-6863d17af3fc None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:34.693316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:34.745741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239793e-9b92-4cfc-8427-6863d17af3fc None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:34.779772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee35f71-6db5-4dd1-83d4-993c32852f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:34.780432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee35f71-6db5-4dd1-83d4-993c32852f97 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:34.780712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee35f71-6db5-4dd1-83d4-993c32852f97 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:34.780984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee35f71-6db5-4dd1-83d4-993c32852f97 None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:34.781809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee35f71-6db5-4dd1-83d4-993c32852f97 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:34.782144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee35f71-6db5-4dd1-83d4-993c32852f97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:34.784191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee35f71-6db5-4dd1-83d4-993c32852f97 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:34.801504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 617/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Fri Jun 3 09:41:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:34.807276 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701e7ba-5475-4e20-ac66-9bdd76d55b47 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:34.905734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701e7ba-5475-4e20-ac66-9bdd76d55b47 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:34.939137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cee77f-8fe4-44c6-8d83-5a9c3c0961aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:34.939915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cee77f-8fe4-44c6-8d83-5a9c3c0961aa None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:34.940204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cee77f-8fe4-44c6-8d83-5a9c3c0961aa None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:34.940483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cee77f-8fe4-44c6-8d83-5a9c3c0961aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:34.950808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cee77f-8fe4-44c6-8d83-5a9c3c0961aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:34.952262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cee77f-8fe4-44c6-8d83-5a9c3c0961aa None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-67cee77f-8fe4-44c6-8d83-5a9c3c0961aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:34.969567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:34.971243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:34.972331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:34.972331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:34.972331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:34.972331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:34.972331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 611/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:34.977171 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee35f71-6db5-4dd1-83d4-993c32852f97 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:35.074979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee35f71-6db5-4dd1-83d4-993c32852f97 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:35.077228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:35.108087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d2293-27b3-406a-8218-597eab771d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:35.108739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d2293-27b3-406a-8218-597eab771d3c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:35.109016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d2293-27b3-406a-8218-597eab771d3c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:35.109292 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d2293-27b3-406a-8218-597eab771d3c None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:35.110052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d2293-27b3-406a-8218-597eab771d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:35.110353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d2293-27b3-406a-8218-597eab771d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:35.112299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d2293-27b3-406a-8218-597eab771d3c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:35.123569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 618/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:35.129863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cee77f-8fe4-44c6-8d83-5a9c3c0961aa None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:35.139065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:35.227171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cee77f-8fe4-44c6-8d83-5a9c3c0961aa None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:35.259676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4645-0ab8-4a08-8b6a-dc42b82622e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:35.260464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4645-0ab8-4a08-8b6a-dc42b82622e7 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:35.260750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4645-0ab8-4a08-8b6a-dc42b82622e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:35.261029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f4645-0ab8-4a08-8b6a-dc42b82622e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:35.267557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4645-0ab8-4a08-8b6a-dc42b82622e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:35.268996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f4645-0ab8-4a08-8b6a-dc42b82622e7 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-382f4645-0ab8-4a08-8b6a-dc42b82622e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:41:35.283488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:35.284529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:35.295112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:41:35.295112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:41:35.295112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:41:35.295112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:41:35.295112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:41:35.295112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:41:35.295112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:35.295112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:35.295112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 612/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:41:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:35.295718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d2293-27b3-406a-8218-597eab771d3c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:35.386949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d2293-27b3-406a-8218-597eab771d3c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:35.420027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0115c-d23c-46ef-aced-d02e095cb676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:35.420670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0115c-d23c-46ef-aced-d02e095cb676 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:35.420952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0115c-d23c-46ef-aced-d02e095cb676 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:35.421230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0115c-d23c-46ef-aced-d02e095cb676 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:35.422041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0115c-d23c-46ef-aced-d02e095cb676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:35.422359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0115c-d23c-46ef-aced-d02e095cb676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:35.424383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b0115c-d23c-46ef-aced-d02e095cb676 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:35.436159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 619/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:41:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:35.442370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4645-0ab8-4a08-8b6a-dc42b82622e7 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:35.508781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:35.542554 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f4645-0ab8-4a08-8b6a-dc42b82622e7 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:35.575460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae6cd3-efc5-48f4-ae54-af66a99183fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:35.576327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae6cd3-efc5-48f4-ae54-af66a99183fe None admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:35.576605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae6cd3-efc5-48f4-ae54-af66a99183fe None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:35.576875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae6cd3-efc5-48f4-ae54-af66a99183fe None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/566c0c4bdcaf4c13b060af4bbff5bc04/roles/4766b6a3fe4746798f255d0813534a04` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:35.590344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae6cd3-efc5-48f4-ae54-af66a99183fe None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=566c0c4bdcaf4c13b060af4bbff5bc04, role_id=4766b6a3fe4746798f255d0813534a04)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:35.591781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae6cd3-efc5-48f4-ae54-af66a99183fe None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:35.690517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 613/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:41:35 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/566c0c4bdcaf4c13b060af4bbff5bc04/roles/4766b6a3fe4746798f255d0813534a04 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:41:37.906543 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d931e3b-3158-410e-8dcf-13e9ef5a9e77 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:37.907062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d931e3b-3158-410e-8dcf-13e9ef5a9e77 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:37.907153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d931e3b-3158-410e-8dcf-13e9ef5a9e77 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:37.908486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 620/1233] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:41:37.914275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fa50e-6890-4903-9c64-dbd2c599bf5b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:37.914702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fa50e-6890-4903-9c64-dbd2c599bf5b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:37.914805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fa50e-6890-4903-9c64-dbd2c599bf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:37.931319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f48fa50e-6890-4903-9c64-dbd2c599bf5b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:41:37.932502 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f48fa50e-6890-4903-9c64-dbd2c599bf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:41:37.998164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 614/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:41:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:41:38.021873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d931e3b-3158-410e-8dcf-13e9ef5a9e77 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:38.047020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:38.122821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d931e3b-3158-410e-8dcf-13e9ef5a9e77 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:38.155636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c260f-5883-469e-a606-21c061b8d156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:38.156415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c260f-5883-469e-a606-21c061b8d156 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:38.156696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c260f-5883-469e-a606-21c061b8d156 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:38.156971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c260f-5883-469e-a606-21c061b8d156 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:38.167035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c260f-5883-469e-a606-21c061b8d156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b85c260f-5883-469e-a606-21c061b8d156 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:41:38.175238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:38.176388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:38.177423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 621/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Fri Jun 3 09:41:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:38.182609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fa50e-6890-4903-9c64-dbd2c599bf5b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:38.227729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:38.276770 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fa50e-6890-4903-9c64-dbd2c599bf5b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:38.308837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a77d9-cf91-424f-b255-117c32525eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:38.309492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a77d9-cf91-424f-b255-117c32525eed None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:38.309773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a77d9-cf91-424f-b255-117c32525eed None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:38.310075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a77d9-cf91-424f-b255-117c32525eed None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:38.310799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a77d9-cf91-424f-b255-117c32525eed None admin] RBAC: Adding query filter params (name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:38.311100 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a77d9-cf91-424f-b255-117c32525eed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:38.312445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a77d9-cf91-424f-b255-117c32525eed None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:38.320361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 615/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Fri Jun 3 09:41:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:38.326084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c260f-5883-469e-a606-21c061b8d156 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:38.407963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:38.424687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c260f-5883-469e-a606-21c061b8d156 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:38.459191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c019c-7277-4660-8b31-b9c702c10dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:38.459983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c019c-7277-4660-8b31-b9c702c10dfd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:38.460265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c019c-7277-4660-8b31-b9c702c10dfd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:38.460540 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c019c-7277-4660-8b31-b9c702c10dfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:38.470569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c019c-7277-4660-8b31-b9c702c10dfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:38.472013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c019c-7277-4660-8b31-b9c702c10dfd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-775c019c-7277-4660-8b31-b9c702c10dfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:38.489133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:38.490248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:38.491442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:38.491442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:38.491442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:38.491442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:38.491442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 622/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:38.495970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a77d9-cf91-424f-b255-117c32525eed None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:38.594113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a77d9-cf91-424f-b255-117c32525eed None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:38.603192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:38.628747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7bcd2-d89d-465d-ab8d-c818c3174fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:38.629415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7bcd2-d89d-465d-ab8d-c818c3174fa6 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:38.629727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7bcd2-d89d-465d-ab8d-c818c3174fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:38.630047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7bcd2-d89d-465d-ab8d-c818c3174fa6 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:38.630759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7bcd2-d89d-465d-ab8d-c818c3174fa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:38.631063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7bcd2-d89d-465d-ab8d-c818c3174fa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:38.632426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7bcd2-d89d-465d-ab8d-c818c3174fa6 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:38.643535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 616/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:38.649334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c019c-7277-4660-8b31-b9c702c10dfd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:38.748068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c019c-7277-4660-8b31-b9c702c10dfd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:38.780973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e872f-d6ce-4f40-8ab6-d2b927481e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:38.781756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e872f-d6ce-4f40-8ab6-d2b927481e6e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:38.782077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e872f-d6ce-4f40-8ab6-d2b927481e6e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:38.782445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e872f-d6ce-4f40-8ab6-d2b927481e6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:38.785312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:38.790001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e872f-d6ce-4f40-8ab6-d2b927481e6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:38.791412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e872f-d6ce-4f40-8ab6-d2b927481e6e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-bf5e872f-d6ce-4f40-8ab6-d2b927481e6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:41:38.806137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:38.807376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:38.808538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 623/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:38.813225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7bcd2-d89d-465d-ab8d-c818c3174fa6 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:38.908345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7bcd2-d89d-465d-ab8d-c818c3174fa6 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:38.942227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427f554-6ab6-4ed3-92a5-496d18df8ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:38.942896 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427f554-6ab6-4ed3-92a5-496d18df8ac8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:38.943182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427f554-6ab6-4ed3-92a5-496d18df8ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:38.943464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427f554-6ab6-4ed3-92a5-496d18df8ac8 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:38.944389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427f554-6ab6-4ed3-92a5-496d18df8ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:38.944702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427f554-6ab6-4ed3-92a5-496d18df8ac8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:38.946161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427f554-6ab6-4ed3-92a5-496d18df8ac8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:38.962825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 617/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Fri Jun 3 09:41:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:38.968776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e872f-d6ce-4f40-8ab6-d2b927481e6e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:39.066788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e872f-d6ce-4f40-8ab6-d2b927481e6e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:39.099099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be616577-6bc2-4f50-a8b2-a46c3713fb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:39.099731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be616577-6bc2-4f50-a8b2-a46c3713fb25 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:39.100015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be616577-6bc2-4f50-a8b2-a46c3713fb25 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:39.100294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be616577-6bc2-4f50-a8b2-a46c3713fb25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:39.160524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:39.166884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be616577-6bc2-4f50-a8b2-a46c3713fb25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:39.168313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be616577-6bc2-4f50-a8b2-a46c3713fb25 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:39.262640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 624/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:41:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:41:39.268588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427f554-6ab6-4ed3-92a5-496d18df8ac8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:39.295944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:39.366817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427f554-6ab6-4ed3-92a5-496d18df8ac8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:39.399774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597f7e9-e2cb-47db-b721-065bf981efcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:39.400549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597f7e9-e2cb-47db-b721-065bf981efcc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:39.400828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597f7e9-e2cb-47db-b721-065bf981efcc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:39.401105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597f7e9-e2cb-47db-b721-065bf981efcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:39.411648 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597f7e9-e2cb-47db-b721-065bf981efcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:39.413099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597f7e9-e2cb-47db-b721-065bf981efcc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-3597f7e9-e2cb-47db-b721-065bf981efcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:39.430585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:39.431629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:39.432713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:39.432713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:39.432713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:39.432713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:39.432713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 618/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:39.437686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be616577-6bc2-4f50-a8b2-a46c3713fb25 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:39.528435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:39.535005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be616577-6bc2-4f50-a8b2-a46c3713fb25 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:39.567826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e5c9d-f040-4a2c-911e-059428cd6816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:39.568479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e5c9d-f040-4a2c-911e-059428cd6816 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:39.568768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e5c9d-f040-4a2c-911e-059428cd6816 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:39.569047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e5c9d-f040-4a2c-911e-059428cd6816 None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:39.569757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e5c9d-f040-4a2c-911e-059428cd6816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:39.570095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e5c9d-f040-4a2c-911e-059428cd6816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:39.571456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e5c9d-f040-4a2c-911e-059428cd6816 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:39.582618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 625/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:39.588247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597f7e9-e2cb-47db-b721-065bf981efcc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:39.664453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:39.684620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597f7e9-e2cb-47db-b721-065bf981efcc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:39.716724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491efac-de37-4f6f-a3ee-76daddf36e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:39.717510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491efac-de37-4f6f-a3ee-76daddf36e48 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:39.717792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491efac-de37-4f6f-a3ee-76daddf36e48 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:39.718092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491efac-de37-4f6f-a3ee-76daddf36e48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:39.725455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491efac-de37-4f6f-a3ee-76daddf36e48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:39.727092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491efac-de37-4f6f-a3ee-76daddf36e48 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-1491efac-de37-4f6f-a3ee-76daddf36e48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:41:39.741016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:39.742075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:39.743303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:41:39.743303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:41:39.743303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:41:39.743303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:41:39.743303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:41:39.743303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:41:39.743303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:39.743303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:39.743303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 619/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:39.748156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e5c9d-f040-4a2c-911e-059428cd6816 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:39.844800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e5c9d-f040-4a2c-911e-059428cd6816 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:39.876436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5943e-2e9e-4b26-a2a9-67b998361983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:39.877080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5943e-2e9e-4b26-a2a9-67b998361983 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:39.877358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5943e-2e9e-4b26-a2a9-67b998361983 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:39.877627 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5943e-2e9e-4b26-a2a9-67b998361983 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:39.878443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5943e-2e9e-4b26-a2a9-67b998361983 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:39.878730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5943e-2e9e-4b26-a2a9-67b998361983 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:39.880144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5943e-2e9e-4b26-a2a9-67b998361983 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:39.893130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 626/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:41:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:39.899272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491efac-de37-4f6f-a3ee-76daddf36e48 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:39.995359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491efac-de37-4f6f-a3ee-76daddf36e48 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:40.028649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa171c-0b87-4b8f-8ace-69ded24d7c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:40.032556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa171c-0b87-4b8f-8ace-69ded24d7c7b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:40.032556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa171c-0b87-4b8f-8ace-69ded24d7c7b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:40.032556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa171c-0b87-4b8f-8ace-69ded24d7c7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:40.032556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa171c-0b87-4b8f-8ace-69ded24d7c7b None admin] RBAC: Adding query filter params (role.id=4766b6a3fe4746798f255d0813534a04, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=566c0c4bdcaf4c13b060af4bbff5bc04) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:40.032556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa171c-0b87-4b8f-8ace-69ded24d7c7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:40.032556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa171c-0b87-4b8f-8ace-69ded24d7c7b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:40.038511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 620/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:41:39 2022] GET /identity/v3/role_assignments?user.id=566c0c4bdcaf4c13b060af4bbff5bc04&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=4766b6a3fe4746798f255d0813534a04 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:42.327469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58ec5c-18a1-4be7-b8f7-839ed4623f45 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:42.328024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58ec5c-18a1-4be7-b8f7-839ed4623f45 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:42.328137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58ec5c-18a1-4be7-b8f7-839ed4623f45 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:42.329409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 627/1247] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:41:42.334823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42b75d-7894-4a1a-bb46-308175d36ea1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:42.335294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42b75d-7894-4a1a-bb46-308175d36ea1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:42.335452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42b75d-7894-4a1a-bb46-308175d36ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:42.340459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:42.352918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9f42b75d-7894-4a1a-bb46-308175d36ea1 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:41:42.354119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9f42b75d-7894-4a1a-bb46-308175d36ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:41:42.417904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 621/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:41:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 09:41:42.441064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58ec5c-18a1-4be7-b8f7-839ed4623f45 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:42.462865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:42.544608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58ec5c-18a1-4be7-b8f7-839ed4623f45 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:42.579214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83515a-1700-4c25-b8e5-07891580fcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:42.579940 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83515a-1700-4c25-b8e5-07891580fcf1 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:42.580219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83515a-1700-4c25-b8e5-07891580fcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:42.580495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83515a-1700-4c25-b8e5-07891580fcf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:42.590283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83515a-1700-4c25-b8e5-07891580fcf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6e83515a-1700-4c25-b8e5-07891580fcf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:41:42.597798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:42.598962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:42.600076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 628/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:41:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:42.605102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42b75d-7894-4a1a-bb46-308175d36ea1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:42.700818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42b75d-7894-4a1a-bb46-308175d36ea1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:42.731715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:42.737166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eaabb-2001-47f4-a6b8-fe5154998bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:42.737838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eaabb-2001-47f4-a6b8-fe5154998bcb None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:42.738151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eaabb-2001-47f4-a6b8-fe5154998bcb None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:42.738431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5eaabb-2001-47f4-a6b8-fe5154998bcb None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:42.739175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5eaabb-2001-47f4-a6b8-fe5154998bcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:42.739481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5eaabb-2001-47f4-a6b8-fe5154998bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:42.741004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5eaabb-2001-47f4-a6b8-fe5154998bcb None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:42.748984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 622/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:41:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:42.754874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83515a-1700-4c25-b8e5-07891580fcf1 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:42.829615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:42.851995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83515a-1700-4c25-b8e5-07891580fcf1 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:42.885423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb160a-36c2-4fc0-ae46-7063aff6b25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:42.886280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb160a-36c2-4fc0-ae46-7063aff6b25e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:42.886561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb160a-36c2-4fc0-ae46-7063aff6b25e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:42.886834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb160a-36c2-4fc0-ae46-7063aff6b25e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:42.897295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb160a-36c2-4fc0-ae46-7063aff6b25e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:42.898765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb160a-36c2-4fc0-ae46-7063aff6b25e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-8acb160a-36c2-4fc0-ae46-7063aff6b25e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:42.916080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:42.917137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:42.918104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:42.918104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:42.918104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:42.918104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:42.918104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 629/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:42.923653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eaabb-2001-47f4-a6b8-fe5154998bcb None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:43.022576 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5eaabb-2001-47f4-a6b8-fe5154998bcb None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:43.056463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a332d7-f9ef-4895-a84f-2ed39d9a75b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:43.057122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a332d7-f9ef-4895-a84f-2ed39d9a75b8 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:43.057407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a332d7-f9ef-4895-a84f-2ed39d9a75b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:43.057718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a332d7-f9ef-4895-a84f-2ed39d9a75b8 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:43.058535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a332d7-f9ef-4895-a84f-2ed39d9a75b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:43.058844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a332d7-f9ef-4895-a84f-2ed39d9a75b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:43.060220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a332d7-f9ef-4895-a84f-2ed39d9a75b8 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:43.071716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 623/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:43.077437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb160a-36c2-4fc0-ae46-7063aff6b25e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:43.176086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb160a-36c2-4fc0-ae46-7063aff6b25e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:43.202289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:43.210420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a45529-1572-48d0-aa63-b3fcd86a8f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:43.211222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a45529-1572-48d0-aa63-b3fcd86a8f5a None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:43.211509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a45529-1572-48d0-aa63-b3fcd86a8f5a None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:43.211792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a45529-1572-48d0-aa63-b3fcd86a8f5a None admin] PATH_INFO: `/v3/users/placement` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:43.219340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a45529-1572-48d0-aa63-b3fcd86a8f5a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:43.221438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a45529-1572-48d0-aa63-b3fcd86a8f5a None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-34a45529-1572-48d0-aa63-b3fcd86a8f5a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:41:43.235480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:43.236588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:43.237698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 630/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:43.242585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a332d7-f9ef-4895-a84f-2ed39d9a75b8 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:43.297270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:43.338708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a332d7-f9ef-4895-a84f-2ed39d9a75b8 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:43.370592 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6945f-a5fd-4e03-aad4-829bc3938297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:43.371256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6945f-a5fd-4e03-aad4-829bc3938297 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:43.371537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6945f-a5fd-4e03-aad4-829bc3938297 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:43.371813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6945f-a5fd-4e03-aad4-829bc3938297 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:43.372630 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6945f-a5fd-4e03-aad4-829bc3938297 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:43.372962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6945f-a5fd-4e03-aad4-829bc3938297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:43.374431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6945f-a5fd-4e03-aad4-829bc3938297 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:43.390561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 624/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Fri Jun 3 09:41:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:43.396911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a45529-1572-48d0-aa63-b3fcd86a8f5a None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:43.496868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a45529-1572-48d0-aa63-b3fcd86a8f5a None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:43.530602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e651115-d417-4120-a378-5b2bb3951c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:43.531247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e651115-d417-4120-a378-5b2bb3951c79 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:43.531531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e651115-d417-4120-a378-5b2bb3951c79 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:43.531811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e651115-d417-4120-a378-5b2bb3951c79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:43.586957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:43.598733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e651115-d417-4120-a378-5b2bb3951c79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:43.600179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e651115-d417-4120-a378-5b2bb3951c79 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:43.693181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 631/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:41:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:41:43.698984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6945f-a5fd-4e03-aad4-829bc3938297 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:43.796001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6945f-a5fd-4e03-aad4-829bc3938297 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:43.808934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:43.830086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec5825-227b-4c3e-9d08-6646e68d4e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:43.830869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec5825-227b-4c3e-9d08-6646e68d4e70 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:43.831156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec5825-227b-4c3e-9d08-6646e68d4e70 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:43.831436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec5825-227b-4c3e-9d08-6646e68d4e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:43.841142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec5825-227b-4c3e-9d08-6646e68d4e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:43.846728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec5825-227b-4c3e-9d08-6646e68d4e70 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-0fec5825-227b-4c3e-9d08-6646e68d4e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:43.859836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:43.861405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:43.862515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:43.862515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:43.862515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:43.862515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:43.862515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 625/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 09:41:43.868897 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e651115-d417-4120-a378-5b2bb3951c79 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:43.958499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:43.970241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e651115-d417-4120-a378-5b2bb3951c79 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:44.004564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb1de8-8f9f-456e-a5a2-519aa61b106e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:44.005242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb1de8-8f9f-456e-a5a2-519aa61b106e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:44.005527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb1de8-8f9f-456e-a5a2-519aa61b106e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:44.005806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb1de8-8f9f-456e-a5a2-519aa61b106e None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:44.006575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb1de8-8f9f-456e-a5a2-519aa61b106e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:44.010396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb1de8-8f9f-456e-a5a2-519aa61b106e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:44.010396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb1de8-8f9f-456e-a5a2-519aa61b106e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:44.019465 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 632/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:44.025113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec5825-227b-4c3e-9d08-6646e68d4e70 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:44.122115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec5825-227b-4c3e-9d08-6646e68d4e70 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:44.155180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e4045-4792-40cd-8836-e9862aa1e6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:44.155961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e4045-4792-40cd-8836-e9862aa1e6bc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:44.156248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e4045-4792-40cd-8836-e9862aa1e6bc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:44.156528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e4045-4792-40cd-8836-e9862aa1e6bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:44.163259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e4045-4792-40cd-8836-e9862aa1e6bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:44.167444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e4045-4792-40cd-8836-e9862aa1e6bc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-318e4045-4792-40cd-8836-e9862aa1e6bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:41:44.178778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:44.179934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:44.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:41:44.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:41:44.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:41:44.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:41:44.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:41:44.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:41:44.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:44.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:44.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 626/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:41:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:44.185770 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb1de8-8f9f-456e-a5a2-519aa61b106e None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:44.289362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb1de8-8f9f-456e-a5a2-519aa61b106e None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:44.322622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981aa288-d6a5-49fb-9b59-99a6a903fd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:44.323297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981aa288-d6a5-49fb-9b59-99a6a903fd89 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:44.323604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981aa288-d6a5-49fb-9b59-99a6a903fd89 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:44.323883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981aa288-d6a5-49fb-9b59-99a6a903fd89 None admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:44.324734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981aa288-d6a5-49fb-9b59-99a6a903fd89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:44.325039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981aa288-d6a5-49fb-9b59-99a6a903fd89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:44.326524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981aa288-d6a5-49fb-9b59-99a6a903fd89 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:44.338476 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 633/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:41:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:44.345022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e4045-4792-40cd-8836-e9862aa1e6bc None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:44.370963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:44.443407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e4045-4792-40cd-8836-e9862aa1e6bc None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:44.476948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf2a0b-702a-4d6d-ab71-91945005e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:44.477590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf2a0b-702a-4d6d-ab71-91945005e465 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:44.477928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf2a0b-702a-4d6d-ab71-91945005e465 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:44.478269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf2a0b-702a-4d6d-ab71-91945005e465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:44.479035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf2a0b-702a-4d6d-ab71-91945005e465 None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=566c0c4bdcaf4c13b060af4bbff5bc04) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:44.479345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf2a0b-702a-4d6d-ab71-91945005e465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:44.480766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf2a0b-702a-4d6d-ab71-91945005e465 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:44.486043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 627/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:41:44 2022] GET /identity/v3/role_assignments?user.id=566c0c4bdcaf4c13b060af4bbff5bc04&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=fe4d636f5c4842f18569347911e6cdec => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:46.716667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8243d-d2e2-414b-aed7-7802034d1d0e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:46.717113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8243d-d2e2-414b-aed7-7802034d1d0e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:46.717228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8243d-d2e2-414b-aed7-7802034d1d0e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:46.718797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 634/1261] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:41:46.724608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a502c6-dbd1-42b9-85b9-c0941eae0357 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:46.725021 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a502c6-dbd1-42b9-85b9-c0941eae0357 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:46.725159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a502c6-dbd1-42b9-85b9-c0941eae0357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:46.742426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-91a502c6-dbd1-42b9-85b9-c0941eae0357 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:41:46.743676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-91a502c6-dbd1-42b9-85b9-c0941eae0357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:41:46.807365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 628/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:41:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:41:46.830844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8243d-d2e2-414b-aed7-7802034d1d0e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:46.841723 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:46.929737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8243d-d2e2-414b-aed7-7802034d1d0e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:46.963911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b1b9-d099-49e8-ba1d-65d07ea1958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:46.964693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b1b9-d099-49e8-ba1d-65d07ea1958c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:46.964979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b1b9-d099-49e8-ba1d-65d07ea1958c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:46.965255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7b1b9-d099-49e8-ba1d-65d07ea1958c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:46.975836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7b1b9-d099-49e8-ba1d-65d07ea1958c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-6ea7b1b9-d099-49e8-ba1d-65d07ea1958c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:41:46.983310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:46.985995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:46.987045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 635/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:41:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:46.990710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a502c6-dbd1-42b9-85b9-c0941eae0357 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:47.008944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:47.084986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a502c6-dbd1-42b9-85b9-c0941eae0357 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:47.116845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68277c8-2160-4cf4-8425-d4d30d2366d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:47.117498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68277c8-2160-4cf4-8425-d4d30d2366d2 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:47.117780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68277c8-2160-4cf4-8425-d4d30d2366d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:47.118088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68277c8-2160-4cf4-8425-d4d30d2366d2 None admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:47.118815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68277c8-2160-4cf4-8425-d4d30d2366d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:47.119115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68277c8-2160-4cf4-8425-d4d30d2366d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:47.120720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68277c8-2160-4cf4-8425-d4d30d2366d2 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:47.128364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 629/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:41:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:47.134180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b1b9-d099-49e8-ba1d-65d07ea1958c None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:47.199341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:47.232800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7b1b9-d099-49e8-ba1d-65d07ea1958c None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:47.265343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2e116-8234-49bd-af96-713ddcb84549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:47.266141 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2e116-8234-49bd-af96-713ddcb84549 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:47.266473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2e116-8234-49bd-af96-713ddcb84549 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:47.266698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2e116-8234-49bd-af96-713ddcb84549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:47.402771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2e116-8234-49bd-af96-713ddcb84549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:47.404229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2e116-8234-49bd-af96-713ddcb84549 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-49b2e116-8234-49bd-af96-713ddcb84549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:47.421198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:47.425176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:47.426181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:47.426181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:47.426181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:47.426181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:47.426181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 636/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:47.428433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68277c8-2160-4cf4-8425-d4d30d2366d2 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:47.506907 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:47.525763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68277c8-2160-4cf4-8425-d4d30d2366d2 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:47.558964 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd1f7e-cd17-4e38-b580-ea22dc89cf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:47.559636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd1f7e-cd17-4e38-b580-ea22dc89cf63 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:47.559922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd1f7e-cd17-4e38-b580-ea22dc89cf63 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:47.560201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd1f7e-cd17-4e38-b580-ea22dc89cf63 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:47.560956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd1f7e-cd17-4e38-b580-ea22dc89cf63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:47.561263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd1f7e-cd17-4e38-b580-ea22dc89cf63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:47.562695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd1f7e-cd17-4e38-b580-ea22dc89cf63 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:47.574866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 630/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:47.581183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2e116-8234-49bd-af96-713ddcb84549 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:47.677122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2e116-8234-49bd-af96-713ddcb84549 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:47.696639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:47.710064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45296b-5982-4635-a947-dd1905dca589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:47.710938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45296b-5982-4635-a947-dd1905dca589 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:47.711220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45296b-5982-4635-a947-dd1905dca589 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:47.711548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45296b-5982-4635-a947-dd1905dca589 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:47.718803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45296b-5982-4635-a947-dd1905dca589 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:47.720319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd45296b-5982-4635-a947-dd1905dca589 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-cd45296b-5982-4635-a947-dd1905dca589 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:41:47.735325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:47.737008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:47.738289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 637/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:47.742906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd1f7e-cd17-4e38-b580-ea22dc89cf63 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:47.837970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd1f7e-cd17-4e38-b580-ea22dc89cf63 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:47.870682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca747a-9f25-4860-9c14-0040dc220841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:47.871341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca747a-9f25-4860-9c14-0040dc220841 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:47.871621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca747a-9f25-4860-9c14-0040dc220841 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:47.871894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca747a-9f25-4860-9c14-0040dc220841 None admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:47.872714 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca747a-9f25-4860-9c14-0040dc220841 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:47.873021 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca747a-9f25-4860-9c14-0040dc220841 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:47.874486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca747a-9f25-4860-9c14-0040dc220841 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:47.883394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:47.891647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 631/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Fri Jun 3 09:41:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:47.897590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45296b-5982-4635-a947-dd1905dca589 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:47.997246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd45296b-5982-4635-a947-dd1905dca589 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:48.029288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0146999-eaaa-4916-b7ea-ed2943f584d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:48.030099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0146999-eaaa-4916-b7ea-ed2943f584d6 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:48.030384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0146999-eaaa-4916-b7ea-ed2943f584d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:48.030660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0146999-eaaa-4916-b7ea-ed2943f584d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:48.041591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0146999-eaaa-4916-b7ea-ed2943f584d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:48.043055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0146999-eaaa-4916-b7ea-ed2943f584d6 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-f0146999-eaaa-4916-b7ea-ed2943f584d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:48.060051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:48.061051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:48.062182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:48.062182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:48.062182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:48.062182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:48.065950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 638/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:48.067197 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca747a-9f25-4860-9c14-0040dc220841 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:48.161763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca747a-9f25-4860-9c14-0040dc220841 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:48.195142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0be55-dcea-4c98-b21a-5fb3b2c667a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:48.195796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0be55-dcea-4c98-b21a-5fb3b2c667a0 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:48.196077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0be55-dcea-4c98-b21a-5fb3b2c667a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:48.196352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0be55-dcea-4c98-b21a-5fb3b2c667a0 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:48.197059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0be55-dcea-4c98-b21a-5fb3b2c667a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:48.197362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0be55-dcea-4c98-b21a-5fb3b2c667a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:48.198752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0be55-dcea-4c98-b21a-5fb3b2c667a0 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:48.209547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 632/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:48.215348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0146999-eaaa-4916-b7ea-ed2943f584d6 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:48.247678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:48.314320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0146999-eaaa-4916-b7ea-ed2943f584d6 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:48.347644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4daa5-57af-46c3-9ba0-3e1b82aaa0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:48.348420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4daa5-57af-46c3-9ba0-3e1b82aaa0a8 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:48.348704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4daa5-57af-46c3-9ba0-3e1b82aaa0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:48.348980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4daa5-57af-46c3-9ba0-3e1b82aaa0a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:48.356430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4daa5-57af-46c3-9ba0-3e1b82aaa0a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:48.357867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4daa5-57af-46c3-9ba0-3e1b82aaa0a8 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-b1e4daa5-57af-46c3-9ba0-3e1b82aaa0a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:41:48.371610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:48.372641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:48.373962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:41:48.373962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:41:48.373962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:41:48.373962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:41:48.373962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:41:48.373962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:41:48.373962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:48.373962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:48.373962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 639/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:41:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:48.378643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0be55-dcea-4c98-b21a-5fb3b2c667a0 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:48.436499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:48.476705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0be55-dcea-4c98-b21a-5fb3b2c667a0 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:48.510134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e08f7-2d42-4b3d-bd37-683adb1c1879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:48.510788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e08f7-2d42-4b3d-bd37-683adb1c1879 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:48.511074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e08f7-2d42-4b3d-bd37-683adb1c1879 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:48.511355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e08f7-2d42-4b3d-bd37-683adb1c1879 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:48.512129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e08f7-2d42-4b3d-bd37-683adb1c1879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:48.512546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e08f7-2d42-4b3d-bd37-683adb1c1879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:48.513994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e08f7-2d42-4b3d-bd37-683adb1c1879 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:48.525785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 633/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:41:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:41:48.531734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4daa5-57af-46c3-9ba0-3e1b82aaa0a8 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:48.622752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:48.631553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4daa5-57af-46c3-9ba0-3e1b82aaa0a8 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:48.663343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f50ce7-7b17-430d-8f75-5f396b1796ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:48.664212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f50ce7-7b17-430d-8f75-5f396b1796ba None admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:48.664493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f50ce7-7b17-430d-8f75-5f396b1796ba None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:48.664766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f50ce7-7b17-430d-8f75-5f396b1796ba None admin] PATH_INFO: `/v3/projects/3c179ab83d804662a720cedf0932336f/users/566c0c4bdcaf4c13b060af4bbff5bc04/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:48.666721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f50ce7-7b17-430d-8f75-5f396b1796ba None admin] RBAC: Authorizing `identity:create_grant(project_id=3c179ab83d804662a720cedf0932336f, user_id=566c0c4bdcaf4c13b060af4bbff5bc04, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:48.668139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f50ce7-7b17-430d-8f75-5f396b1796ba None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:48.808268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 640/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Fri Jun 3 09:41:48 2022] PUT /identity/v3/projects/3c179ab83d804662a720cedf0932336f/users/566c0c4bdcaf4c13b060af4bbff5bc04/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:41:50.994269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c67e5-275e-4fbd-ba06-e4ad4492c2d9 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:50.994766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c67e5-275e-4fbd-ba06-e4ad4492c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:50.994766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c67e5-275e-4fbd-ba06-e4ad4492c2d9 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:50.996228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 634/1274] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:41:51.001251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e548-7daf-4dcb-bb00-e30f74500e3f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:51.001625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e548-7daf-4dcb-bb00-e30f74500e3f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:51.002358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01e548-7daf-4dcb-bb00-e30f74500e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:51.019117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2f01e548-7daf-4dcb-bb00-e30f74500e3f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:41:51.020278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2f01e548-7daf-4dcb-bb00-e30f74500e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:41:51.084233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 641/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:41:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:41:51.107201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c67e5-275e-4fbd-ba06-e4ad4492c2d9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:51.210461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c67e5-275e-4fbd-ba06-e4ad4492c2d9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:51.227228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:51.244028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4d9bd-21d1-40c1-803b-edf88f02c633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:51.244858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4d9bd-21d1-40c1-803b-edf88f02c633 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:51.245100 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4d9bd-21d1-40c1-803b-edf88f02c633 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:51.245382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4d9bd-21d1-40c1-803b-edf88f02c633 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:51.255781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4d9bd-21d1-40c1-803b-edf88f02c633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-d7a4d9bd-21d1-40c1-803b-edf88f02c633 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 09:41:51.263750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:51.264909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:51.265823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 635/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Fri Jun 3 09:41:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:51.271685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e548-7daf-4dcb-bb00-e30f74500e3f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:51.352168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:51.369968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01e548-7daf-4dcb-bb00-e30f74500e3f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:51.402506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e83097-f9e7-45d0-952b-9c55d015abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:51.403165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e83097-f9e7-45d0-952b-9c55d015abd5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:51.403449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e83097-f9e7-45d0-952b-9c55d015abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:51.403726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e83097-f9e7-45d0-952b-9c55d015abd5 None admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:51.404465 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e83097-f9e7-45d0-952b-9c55d015abd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:51.404773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e83097-f9e7-45d0-952b-9c55d015abd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:51.406172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e83097-f9e7-45d0-952b-9c55d015abd5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:51.414571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 642/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Fri Jun 3 09:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:51.420345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4d9bd-21d1-40c1-803b-edf88f02c633 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:51.519534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4d9bd-21d1-40c1-803b-edf88f02c633 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:51.552687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9179a-12ce-44e9-9f80-4eee83acde5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:51.553470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9179a-12ce-44e9-9f80-4eee83acde5b None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:51.553758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9179a-12ce-44e9-9f80-4eee83acde5b None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:51.554064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9179a-12ce-44e9-9f80-4eee83acde5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:51.565024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9179a-12ce-44e9-9f80-4eee83acde5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:51.566501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9179a-12ce-44e9-9f80-4eee83acde5b None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f8d9179a-12ce-44e9-9f80-4eee83acde5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:51.583291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:51.584380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:51.585624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:51.585624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:51.585624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:51.585624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:51.585624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 636/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:51.590412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e83097-f9e7-45d0-952b-9c55d015abd5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:51.689835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e83097-f9e7-45d0-952b-9c55d015abd5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:51.723441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e05b1f-0f93-44c8-928b-5d61861c81dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:51.724102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e05b1f-0f93-44c8-928b-5d61861c81dd None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:51.724384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e05b1f-0f93-44c8-928b-5d61861c81dd None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:51.724659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e05b1f-0f93-44c8-928b-5d61861c81dd None admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:51.725376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e05b1f-0f93-44c8-928b-5d61861c81dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:51.725680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e05b1f-0f93-44c8-928b-5d61861c81dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:51.727132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e05b1f-0f93-44c8-928b-5d61861c81dd None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:51.739205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 643/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:51.745034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9179a-12ce-44e9-9f80-4eee83acde5b None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:51.774713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:51.840550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9179a-12ce-44e9-9f80-4eee83acde5b None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:51.872971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f7b6f-a28e-4ee8-b67c-e052419e8196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:51.873741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f7b6f-a28e-4ee8-b67c-e052419e8196 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:51.874044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f7b6f-a28e-4ee8-b67c-e052419e8196 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:51.874317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f7b6f-a28e-4ee8-b67c-e052419e8196 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:51.881519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f7b6f-a28e-4ee8-b67c-e052419e8196 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:51.883057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f7b6f-a28e-4ee8-b67c-e052419e8196 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-078f7b6f-a28e-4ee8-b67c-e052419e8196 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 09:41:51.897313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:51.898484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:51.899688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 637/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:51.904525 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e05b1f-0f93-44c8-928b-5d61861c81dd None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:51.908750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:51.999567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e05b1f-0f93-44c8-928b-5d61861c81dd None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:52.032121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308df9f-ebb9-4cb1-b6ac-4e8010dc973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:52.032784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308df9f-ebb9-4cb1-b6ac-4e8010dc973f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:52.033605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308df9f-ebb9-4cb1-b6ac-4e8010dc973f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:52.033909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308df9f-ebb9-4cb1-b6ac-4e8010dc973f None admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:52.037280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308df9f-ebb9-4cb1-b6ac-4e8010dc973f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:52.037280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308df9f-ebb9-4cb1-b6ac-4e8010dc973f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:52.037280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308df9f-ebb9-4cb1-b6ac-4e8010dc973f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:52.055856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 644/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Fri Jun 3 09:41:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:41:52.061702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f7b6f-a28e-4ee8-b67c-e052419e8196 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:52.149903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:52.159392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f7b6f-a28e-4ee8-b67c-e052419e8196 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:52.190981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3ab82-6cab-4417-940a-cb4af7d0aa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:52.191624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3ab82-6cab-4417-940a-cb4af7d0aa5d None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:52.191908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3ab82-6cab-4417-940a-cb4af7d0aa5d None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:52.192185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3ab82-6cab-4417-940a-cb4af7d0aa5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:52.257277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3ab82-6cab-4417-940a-cb4af7d0aa5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:52.258747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3ab82-6cab-4417-940a-cb4af7d0aa5d None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:52.342847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:52.353055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 638/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Fri Jun 3 09:41:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 09:41:52.358853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308df9f-ebb9-4cb1-b6ac-4e8010dc973f None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:52.418736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:52.455178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308df9f-ebb9-4cb1-b6ac-4e8010dc973f None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:52.487845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfd0d4-20e0-4ea5-a2c1-d72b721d4ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:52.488763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfd0d4-20e0-4ea5-a2c1-d72b721d4ae5 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:52.489048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfd0d4-20e0-4ea5-a2c1-d72b721d4ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:52.489381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfd0d4-20e0-4ea5-a2c1-d72b721d4ae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:52.499035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfd0d4-20e0-4ea5-a2c1-d72b721d4ae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:52.500571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfd0d4-20e0-4ea5-a2c1-d72b721d4ae5 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-f1bfd0d4-20e0-4ea5-a2c1-d72b721d4ae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 09:41:52.518754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:52.519779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:52.521087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 09:41:52.521087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 09:41:52.521087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 09:41:52.521087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:52.521087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 645/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:52.525571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3ab82-6cab-4417-940a-cb4af7d0aa5d None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:52.622421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3ab82-6cab-4417-940a-cb4af7d0aa5d None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:52.654757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09cb65-76e0-430a-8007-00e9ad428b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:52.655450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09cb65-76e0-430a-8007-00e9ad428b98 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:52.655726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09cb65-76e0-430a-8007-00e9ad428b98 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:52.655999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09cb65-76e0-430a-8007-00e9ad428b98 None admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:52.656702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09cb65-76e0-430a-8007-00e9ad428b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:52.657004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09cb65-76e0-430a-8007-00e9ad428b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:52.658905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09cb65-76e0-430a-8007-00e9ad428b98 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:52.669874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 639/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Fri Jun 3 09:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:52.676105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfd0d4-20e0-4ea5-a2c1-d72b721d4ae5 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:52.772159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfd0d4-20e0-4ea5-a2c1-d72b721d4ae5 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:52.786319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:52.805747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bda21c-f143-48e8-baf1-359764db8ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:52.806606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bda21c-f143-48e8-baf1-359764db8ec0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:52.806892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bda21c-f143-48e8-baf1-359764db8ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:52.807167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bda21c-f143-48e8-baf1-359764db8ec0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:52.813600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bda21c-f143-48e8-baf1-359764db8ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:52.815123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bda21c-f143-48e8-baf1-359764db8ec0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application [None req-18bda21c-f143-48e8-baf1-359764db8ec0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 09:41:52.828946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:52.830506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:52.831513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 09:41:52.831513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 09:41:52.831513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 09:41:52.831513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 09:41:52.831513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 09:41:52.831513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 09:41:52.831513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 09:41:52.831513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: ERROR keystone.server.flask.application Jun 03 09:41:52.831513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 646/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Fri Jun 3 09:41:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:52.836758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09cb65-76e0-430a-8007-00e9ad428b98 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:52.886511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:52.936339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09cb65-76e0-430a-8007-00e9ad428b98 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:52.968500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bc44a-555d-4dfc-a1e2-7d5ec1b47971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:52.969163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bc44a-555d-4dfc-a1e2-7d5ec1b47971 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:52.973818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bc44a-555d-4dfc-a1e2-7d5ec1b47971 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:52.973818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bc44a-555d-4dfc-a1e2-7d5ec1b47971 None admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:52.973818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bc44a-555d-4dfc-a1e2-7d5ec1b47971 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:52.973818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bc44a-555d-4dfc-a1e2-7d5ec1b47971 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:52.973818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bc44a-555d-4dfc-a1e2-7d5ec1b47971 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:52.984538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 640/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Fri Jun 3 09:41:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:52.990661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bda21c-f143-48e8-baf1-359764db8ec0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:53.087203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bda21c-f143-48e8-baf1-359764db8ec0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:53.120065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520067cb-7247-4c35-9349-21a4b529ea2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:53.120703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520067cb-7247-4c35-9349-21a4b529ea2c None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:53.121020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520067cb-7247-4c35-9349-21a4b529ea2c None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:53.121293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520067cb-7247-4c35-9349-21a4b529ea2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:53.122193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520067cb-7247-4c35-9349-21a4b529ea2c None admin] RBAC: Adding query filter params (role.id=fe4d636f5c4842f18569347911e6cdec, scope.project.id=3c179ab83d804662a720cedf0932336f, user.id=566c0c4bdcaf4c13b060af4bbff5bc04) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:53.122507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520067cb-7247-4c35-9349-21a4b529ea2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:53.123926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520067cb-7247-4c35-9349-21a4b529ea2c None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:53.129709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 647/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Fri Jun 3 09:41:52 2022] GET /identity/v3/role_assignments?user.id=566c0c4bdcaf4c13b060af4bbff5bc04&scope.project.id=3c179ab83d804662a720cedf0932336f&role.id=fe4d636f5c4842f18569347911e6cdec => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:55.410204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ff598-0900-4e6d-b693-729be3eac897 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:55.410735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ff598-0900-4e6d-b693-729be3eac897 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:55.410874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ff598-0900-4e6d-b693-729be3eac897 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:55.412128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 641/1288] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:41:55.417542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6f88e-c30d-4196-a491-2a4783d74ec0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:55.417989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6f88e-c30d-4196-a491-2a4783d74ec0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:55.418101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6f88e-c30d-4196-a491-2a4783d74ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:55.434860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fca6f88e-c30d-4196-a491-2a4783d74ec0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:41:55.436064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fca6f88e-c30d-4196-a491-2a4783d74ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:41:55.465345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:55.501512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 648/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:41:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:41:55.524034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ff598-0900-4e6d-b693-729be3eac897 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:55.623715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ff598-0900-4e6d-b693-729be3eac897 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:55.634397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:55.657667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce73e7-c732-4064-bf36-004273954f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:55.659038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce73e7-c732-4064-bf36-004273954f5e None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:55.659489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce73e7-c732-4064-bf36-004273954f5e None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:55.660565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce73e7-c732-4064-bf36-004273954f5e None admin] PATH_INFO: `/v3/services/placement` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:55.661207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce73e7-c732-4064-bf36-004273954f5e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:55.662897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce73e7-c732-4064-bf36-004273954f5e None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-66ce73e7-c732-4064-bf36-004273954f5e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:41:55.675454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:41:55.676478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:41:55.677378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 642/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:41:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:41:55.685759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6f88e-c30d-4196-a491-2a4783d74ec0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:55.778957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6f88e-c30d-4196-a491-2a4783d74ec0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:55.813307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a676bf4-4f96-4066-9d78-967ee24cfe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:55.813983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a676bf4-4f96-4066-9d78-967ee24cfe8e None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:55.814290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a676bf4-4f96-4066-9d78-967ee24cfe8e None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:55.814561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a676bf4-4f96-4066-9d78-967ee24cfe8e None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:55.815249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a676bf4-4f96-4066-9d78-967ee24cfe8e None admin] RBAC: Adding query filter params (name=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:55.815552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a676bf4-4f96-4066-9d78-967ee24cfe8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:55.816904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a676bf4-4f96-4066-9d78-967ee24cfe8e None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:55.821569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 649/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Fri Jun 3 09:41:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:41:55.827401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce73e7-c732-4064-bf36-004273954f5e None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:55.924966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce73e7-c732-4064-bf36-004273954f5e None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:55.958133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04bd67-6aff-4b62-9e4f-4ba5835f4dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:55.958794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04bd67-6aff-4b62-9e4f-4ba5835f4dcc None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:55.959081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04bd67-6aff-4b62-9e4f-4ba5835f4dcc None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:55.959362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04bd67-6aff-4b62-9e4f-4ba5835f4dcc None admin] PATH_INFO: `/v3/services` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:55.960067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04bd67-6aff-4b62-9e4f-4ba5835f4dcc None admin] RBAC: Adding query filter params (type=placement) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:41:55.960377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04bd67-6aff-4b62-9e4f-4ba5835f4dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:55.961931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04bd67-6aff-4b62-9e4f-4ba5835f4dcc None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:55.966489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 643/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Fri Jun 3 09:41:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:41:58.217979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196d25d-64f0-4e2f-b83c-3742551866b1 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:58.218532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196d25d-64f0-4e2f-b83c-3742551866b1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:58.218532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196d25d-64f0-4e2f-b83c-3742551866b1 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:58.219873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 650/1293] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:41:58.225040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42593d-3e3e-4f80-aa64-6d626ac0795b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:58.225442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42593d-3e3e-4f80-aa64-6d626ac0795b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:58.225596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42593d-3e3e-4f80-aa64-6d626ac0795b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:58.242431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3d42593d-3e3e-4f80-aa64-6d626ac0795b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:41:58.243591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3d42593d-3e3e-4f80-aa64-6d626ac0795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:41:58.272287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:58.307546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 644/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:41:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:41:58.330498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196d25d-64f0-4e2f-b83c-3742551866b1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:41:58.360475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:41:58.427947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196d25d-64f0-4e2f-b83c-3742551866b1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:41:58.460018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359b6a5-3c1c-4528-8944-19ab8f74c207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:41:58.460664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359b6a5-3c1c-4528-8944-19ab8f74c207 None admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:41:58.460951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359b6a5-3c1c-4528-8944-19ab8f74c207 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:41:58.461227 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359b6a5-3c1c-4528-8944-19ab8f74c207 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:41:58.461970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359b6a5-3c1c-4528-8944-19ab8f74c207 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:41:58.463316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359b6a5-3c1c-4528-8944-19ab8f74c207 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:41:58.568430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 651/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Fri Jun 3 09:41:58 2022] POST /identity/v3/services => generated 246 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:42:00.818372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4a9c3-3cae-4ab3-bb5e-a6d7e683ef58 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:00.818949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4a9c3-3cae-4ab3-bb5e-a6d7e683ef58 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:00.818949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4a9c3-3cae-4ab3-bb5e-a6d7e683ef58 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:00.820340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 645/1296] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:42:00.825573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735be451-cbfc-43c3-8440-8a8d89056b38 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:00.826032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735be451-cbfc-43c3-8440-8a8d89056b38 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:00.826164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735be451-cbfc-43c3-8440-8a8d89056b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:00.842476 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-735be451-cbfc-43c3-8440-8a8d89056b38 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:42:00.843666 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-735be451-cbfc-43c3-8440-8a8d89056b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:42:00.914507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 652/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:42:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:42:00.937656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4a9c3-3cae-4ab3-bb5e-a6d7e683ef58 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:42:01.036133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4a9c3-3cae-4ab3-bb5e-a6d7e683ef58 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:42:01.069176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ca8e0-48c8-4ead-bd4d-c017331a3419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:42:01.069994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ca8e0-48c8-4ead-bd4d-c017331a3419 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:01.070499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ca8e0-48c8-4ead-bd4d-c017331a3419 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:01.070703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ca8e0-48c8-4ead-bd4d-c017331a3419 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:01.071351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ca8e0-48c8-4ead-bd4d-c017331a3419 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:42:01.072723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ca8e0-48c8-4ead-bd4d-c017331a3419 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-f32ca8e0-48c8-4ead-bd4d-c017331a3419 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:42:01.085166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:42:01.086456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:42:01.087768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 646/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:42:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:42:01.092457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735be451-cbfc-43c3-8440-8a8d89056b38 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:42:01.110626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:42:01.187889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735be451-cbfc-43c3-8440-8a8d89056b38 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:42:01.219862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997152a-fbd8-448d-aaec-64af7c0015a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:42:01.221355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997152a-fbd8-448d-aaec-64af7c0015a0 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:01.221355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997152a-fbd8-448d-aaec-64af7c0015a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:01.221355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997152a-fbd8-448d-aaec-64af7c0015a0 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:01.221748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997152a-fbd8-448d-aaec-64af7c0015a0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:42:01.222080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997152a-fbd8-448d-aaec-64af7c0015a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:42:01.223431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c997152a-fbd8-448d-aaec-64af7c0015a0 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:42:01.228339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 653/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Fri Jun 3 09:42:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:42:01.234218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ca8e0-48c8-4ead-bd4d-c017331a3419 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:42:01.266552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:42:01.333923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ca8e0-48c8-4ead-bd4d-c017331a3419 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:42:01.367533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f7ee-aa4f-4dd4-ab17-31adcd318b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:42:01.368183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f7ee-aa4f-4dd4-ab17-31adcd318b00 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:01.368471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f7ee-aa4f-4dd4-ab17-31adcd318b00 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:01.368752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f7ee-aa4f-4dd4-ab17-31adcd318b00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:01.369602 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42f7ee-aa4f-4dd4-ab17-31adcd318b00 None admin] RBAC: Adding query filter params (interface=public, service_id=34e122907c194f7394afe6bea3461e26, region_id=RegionOne) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:42:01.369942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42f7ee-aa4f-4dd4-ab17-31adcd318b00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:42:01.371309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42f7ee-aa4f-4dd4-ab17-31adcd318b00 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:42:01.376321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 647/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Fri Jun 3 09:42:01 2022] GET /identity/v3/endpoints?service_id=34e122907c194f7394afe6bea3461e26&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:42:01.382650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997152a-fbd8-448d-aaec-64af7c0015a0 None admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:42:01.463052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:42:01.484653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997152a-fbd8-448d-aaec-64af7c0015a0 None admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:42:01.518265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde70789-b01c-4b1e-944f-33816300260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:42:01.518925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde70789-b01c-4b1e-944f-33816300260f None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:01.519228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde70789-b01c-4b1e-944f-33816300260f None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:01.519529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde70789-b01c-4b1e-944f-33816300260f None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:01.520192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde70789-b01c-4b1e-944f-33816300260f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:42:01.521552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde70789-b01c-4b1e-944f-33816300260f None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:42:01.528442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 654/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Fri Jun 3 09:42:01 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:42:03.777840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232d4e9-8262-4190-bfa5-465f226ebdba None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:03.778370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232d4e9-8262-4190-bfa5-465f226ebdba None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:03.778464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232d4e9-8262-4190-bfa5-465f226ebdba None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:03.779827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 648/1302] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:42:03.785838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ae74-69c3-40c4-9f9c-18c5b87491b4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:03.786268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ae74-69c3-40c4-9f9c-18c5b87491b4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:03.794049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8ae74-69c3-40c4-9f9c-18c5b87491b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:03.803556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b6c8ae74-69c3-40c4-9f9c-18c5b87491b4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:42:03.804740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b6c8ae74-69c3-40c4-9f9c-18c5b87491b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:42:03.869678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 655/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:42:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 09:42:03.892608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232d4e9-8262-4190-bfa5-465f226ebdba None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:42:03.985323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:42:03.992453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232d4e9-8262-4190-bfa5-465f226ebdba None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:42:04.024480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274882d-cba8-4890-9959-2100859e5b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:42:04.025271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274882d-cba8-4890-9959-2100859e5b85 None admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:04.025558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274882d-cba8-4890-9959-2100859e5b85 None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:04.025836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274882d-cba8-4890-9959-2100859e5b85 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:04.026506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274882d-cba8-4890-9959-2100859e5b85 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:42:04.027868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274882d-cba8-4890-9959-2100859e5b85 None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application [None req-8274882d-cba8-4890-9959-2100859e5b85 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 09:42:04.040306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 09:42:04.047138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application with Lock( Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self._enter() Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application return self.creator() Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application created_value = creator( Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: ERROR keystone.server.flask.application Jun 03 09:42:04.048255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 649/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Fri Jun 3 09:42:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 09:42:04.049758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8ae74-69c3-40c4-9f9c-18c5b87491b4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:42:04.122812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:42:04.144974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8ae74-69c3-40c4-9f9c-18c5b87491b4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:42:04.178267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539bfce-480d-42a2-a3c7-a78480393e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:42:04.178925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539bfce-480d-42a2-a3c7-a78480393e17 None admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:04.179202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539bfce-480d-42a2-a3c7-a78480393e17 None admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:04.179475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539bfce-480d-42a2-a3c7-a78480393e17 None admin] PATH_INFO: `/v3/services` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:04.180175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539bfce-480d-42a2-a3c7-a78480393e17 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:42:04.180477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539bfce-480d-42a2-a3c7-a78480393e17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:42:04.181820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539bfce-480d-42a2-a3c7-a78480393e17 None admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:42:04.186753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 656/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Fri Jun 3 09:42:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:42:04.192643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274882d-cba8-4890-9959-2100859e5b85 None admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:42:04.289435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274882d-cba8-4890-9959-2100859e5b85 None admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:42:04.321711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816d098-0b17-45b6-8cde-c4e1dafbf0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:42:04.322396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d098-0b17-45b6-8cde-c4e1dafbf0db None admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:42:04.322682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d098-0b17-45b6-8cde-c4e1dafbf0db None admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:42:04.322959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d098-0b17-45b6-8cde-c4e1dafbf0db None admin] PATH_INFO: `/v3/endpoints` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:42:04.323706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816d098-0b17-45b6-8cde-c4e1dafbf0db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:42:04.325067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816d098-0b17-45b6-8cde-c4e1dafbf0db None admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:42:04.396822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 650/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Fri Jun 3 09:42:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:42:14.193441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=30 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:42:14.391890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:39.363177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782206de-959c-4293-9bdc-5ca83d28386b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:39.363789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782206de-959c-4293-9bdc-5ca83d28386b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:39.363789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782206de-959c-4293-9bdc-5ca83d28386b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:39.365381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 657/1307] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:49:39.370667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98582580-16d8-4a53-a373-7b528ff294b3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:39.371150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98582580-16d8-4a53-a373-7b528ff294b3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:39.371242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98582580-16d8-4a53-a373-7b528ff294b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:39.580481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-98582580-16d8-4a53-a373-7b528ff294b3 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:49:39.581693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-98582580-16d8-4a53-a373-7b528ff294b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:39.714447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 651/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:49:39.753767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170da19-2637-435d-9a6d-6b288be90fc0 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:39.754042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170da19-2637-435d-9a6d-6b288be90fc0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:39.754317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170da19-2637-435d-9a6d-6b288be90fc0 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:39.755895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 658/1309] 10.222.0.42 () {54 vars in 964 bytes} [Fri Jun 3 09:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:49:39.768894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c579981-2f20-455c-a781-98312073d310 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:39.769909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c579981-2f20-455c-a781-98312073d310 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:39.770690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c579981-2f20-455c-a781-98312073d310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:39.885577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3c579981-2f20-455c-a781-98312073d310 None None] MFA Rules not processed for user `362797538cbd48dc9c73b09b0548959e`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:49:39.887065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3c579981-2f20-455c-a781-98312073d310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:39.909645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:39.974432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170da19-2637-435d-9a6d-6b288be90fc0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:49:39.976143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b170da19-2637-435d-9a6d-6b288be90fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:39.982056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 652/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Fri Jun 3 09:49:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:49:40.093749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170da19-2637-435d-9a6d-6b288be90fc0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:49:40.170170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f6d2c-a2ea-48ee-9185-dc6101cdc422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:49:40.170738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f6d2c-a2ea-48ee-9185-dc6101cdc422 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:40.171017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f6d2c-a2ea-48ee-9185-dc6101cdc422 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:40.171293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f6d2c-a2ea-48ee-9185-dc6101cdc422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:40.172917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ad0f6d2c-a2ea-48ee-9185-dc6101cdc422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:40.232811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:40.290287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f6d2c-a2ea-48ee-9185-dc6101cdc422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:49:40.291732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f6d2c-a2ea-48ee-9185-dc6101cdc422 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:49:40.312472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 659/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:49:42.848410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baeb1a-bc27-49b5-a694-14c164d07fda None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:42.848890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baeb1a-bc27-49b5-a694-14c164d07fda None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:42.848989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baeb1a-bc27-49b5-a694-14c164d07fda None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:42.850390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 653/1312] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:49:42.855963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4fdef-6d4a-452b-b4b8-15ecdb000a40 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:42.856463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4fdef-6d4a-452b-b4b8-15ecdb000a40 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:42.856558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4fdef-6d4a-452b-b4b8-15ecdb000a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:42.957161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-abe4fdef-6d4a-452b-b4b8-15ecdb000a40 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:49:42.958534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-abe4fdef-6d4a-452b-b4b8-15ecdb000a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:43.061904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 660/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:49:43.099417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baeb1a-bc27-49b5-a694-14c164d07fda None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:49:43.100933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-65baeb1a-bc27-49b5-a694-14c164d07fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:43.210778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baeb1a-bc27-49b5-a694-14c164d07fda None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:49:43.282101 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:43.284845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa0a75-40e0-4ce5-ba28-7849dab138f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:49:43.285794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa0a75-40e0-4ce5-ba28-7849dab138f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:43.286158 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa0a75-40e0-4ce5-ba28-7849dab138f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:43.286495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa0a75-40e0-4ce5-ba28-7849dab138f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:43.288194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cafa0a75-40e0-4ce5-ba28-7849dab138f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:43.401486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa0a75-40e0-4ce5-ba28-7849dab138f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:49:43.403111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa0a75-40e0-4ce5-ba28-7849dab138f7 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:49:43.423155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 654/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:49:46.194552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae4ccd-f43b-466d-8352-850eef436392 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:46.195040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae4ccd-f43b-466d-8352-850eef436392 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:46.195263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae4ccd-f43b-466d-8352-850eef436392 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:46.196705 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 661/1315] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:49:46.202499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b95e5-d3e1-4c23-b0a4-9e3ba9091532 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:46.202935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b95e5-d3e1-4c23-b0a4-9e3ba9091532 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:46.203041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b95e5-d3e1-4c23-b0a4-9e3ba9091532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:46.266455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b69b95e5-d3e1-4c23-b0a4-9e3ba9091532 None None] MFA Rules not processed for user `c987d6e5675d41e7b4123050fafb4d99`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:49:46.267686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b69b95e5-d3e1-4c23-b0a4-9e3ba9091532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:46.317328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:46.360115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 655/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:49:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:49:46.400338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae4ccd-f43b-466d-8352-850eef436392 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:49:46.401844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-38ae4ccd-f43b-466d-8352-850eef436392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:46.411432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:46.502570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae4ccd-f43b-466d-8352-850eef436392 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:49:46.572004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e51831-4f2b-45da-a20a-e45471a823d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:49:46.572631 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e51831-4f2b-45da-a20a-e45471a823d3 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:46.572913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e51831-4f2b-45da-a20a-e45471a823d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:46.573195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e51831-4f2b-45da-a20a-e45471a823d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:46.574828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-43e51831-4f2b-45da-a20a-e45471a823d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:46.634945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:46.680586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e51831-4f2b-45da-a20a-e45471a823d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:49:46.681967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e51831-4f2b-45da-a20a-e45471a823d3 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:49:46.703306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 662/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:49:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:49:49.689820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523d96d-0b44-4bd6-ae4e-14fa5b76a416 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:49.690367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523d96d-0b44-4bd6-ae4e-14fa5b76a416 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:49.690481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523d96d-0b44-4bd6-ae4e-14fa5b76a416 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:49.691781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 656/1318] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:49:49.697029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ac12-395c-48e3-afe1-2b5e459d79bb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:49.697465 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ac12-395c-48e3-afe1-2b5e459d79bb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:49.697592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5ac12-395c-48e3-afe1-2b5e459d79bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:49.762636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5ef5ac12-395c-48e3-afe1-2b5e459d79bb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:49:49.763819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5ef5ac12-395c-48e3-afe1-2b5e459d79bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:49.860461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 663/1319] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:49:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:49:49.914192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f513e-19bc-4146-8191-752366be8669 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:49.914476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f513e-19bc-4146-8191-752366be8669 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:49.914751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f513e-19bc-4146-8191-752366be8669 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:49.916126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 657/1320] 10.222.0.42 () {54 vars in 964 bytes} [Fri Jun 3 09:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:49:49.922687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f918803-1cf8-44ea-ab89-c3325ffc1dc6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:49.922978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f918803-1cf8-44ea-ab89-c3325ffc1dc6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:49.923262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f918803-1cf8-44ea-ab89-c3325ffc1dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:49.943252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:49.996686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7f918803-1cf8-44ea-ab89-c3325ffc1dc6 None None] MFA Rules not processed for user `362797538cbd48dc9c73b09b0548959e`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:49:49.998452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7f918803-1cf8-44ea-ab89-c3325ffc1dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:50.071617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 664/1321] 10.222.0.42 () {58 vars in 1061 bytes} [Fri Jun 3 09:49:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:49:50.081167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f513e-19bc-4146-8191-752366be8669 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:49:50.082705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bd9f513e-19bc-4146-8191-752366be8669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:50.181626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f513e-19bc-4146-8191-752366be8669 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:49:50.254156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e109fb-0861-418b-8b1e-d826449f9a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:49:50.254800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e109fb-0861-418b-8b1e-d826449f9a35 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:50.255080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e109fb-0861-418b-8b1e-d826449f9a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:50.255352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e109fb-0861-418b-8b1e-d826449f9a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:50.256972 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-07e109fb-0861-418b-8b1e-d826449f9a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:50.263731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:50.361965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e109fb-0861-418b-8b1e-d826449f9a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:49:50.363398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e109fb-0861-418b-8b1e-d826449f9a35 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:49:50.381383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 658/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:49:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:49:54.122277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657376-a120-42fc-b051-6b2520ab49dd None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:54.123432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657376-a120-42fc-b051-6b2520ab49dd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:54.123432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657376-a120-42fc-b051-6b2520ab49dd None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:54.124765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 665/1323] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:49:54.130033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606040a6-d496-402d-b65e-2f2d3a8f5ccf None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:54.130474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606040a6-d496-402d-b65e-2f2d3a8f5ccf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:54.130589 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606040a6-d496-402d-b65e-2f2d3a8f5ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:54.195365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-606040a6-d496-402d-b65e-2f2d3a8f5ccf None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:49:54.196556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-606040a6-d496-402d-b65e-2f2d3a8f5ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:54.253746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:54.291718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 659/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:49:54.327428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a657376-a120-42fc-b051-6b2520ab49dd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:49:54.328980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0a657376-a120-42fc-b051-6b2520ab49dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:54.432607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a657376-a120-42fc-b051-6b2520ab49dd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:49:54.445195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:54.507036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fa936-74db-47da-9af0-f4950d7c21f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:49:54.507693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fa936-74db-47da-9af0-f4950d7c21f1 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:54.507977 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fa936-74db-47da-9af0-f4950d7c21f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:54.508254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fa936-74db-47da-9af0-f4950d7c21f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:54.509927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a59fa936-74db-47da-9af0-f4950d7c21f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:54.621840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fa936-74db-47da-9af0-f4950d7c21f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:49:54.623852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fa936-74db-47da-9af0-f4950d7c21f1 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:49:54.642638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 666/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:49:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:49:56.949245 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a22b6-0a7c-48db-929a-e97af0674079 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:56.949793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a22b6-0a7c-48db-929a-e97af0674079 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:56.949793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a22b6-0a7c-48db-929a-e97af0674079 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:56.951257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 660/1326] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:49:56.956495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2292f-da80-4dac-9fcc-8393e25bddf5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:56.956954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2292f-da80-4dac-9fcc-8393e25bddf5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:56.957095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2292f-da80-4dac-9fcc-8393e25bddf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:57.003437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:57.020894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-cda2292f-da80-4dac-9fcc-8393e25bddf5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:49:57.028972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cda2292f-da80-4dac-9fcc-8393e25bddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:57.118678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 667/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:49:57.155183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a22b6-0a7c-48db-929a-e97af0674079 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:49:57.156698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-919a22b6-0a7c-48db-929a-e97af0674079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:57.261417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a22b6-0a7c-48db-929a-e97af0674079 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:49:57.334774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c0587-1ef3-4e2a-8ae1-249f77b9fd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:49:57.335430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c0587-1ef3-4e2a-8ae1-249f77b9fd32 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:57.335718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c0587-1ef3-4e2a-8ae1-249f77b9fd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:57.336001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c0587-1ef3-4e2a-8ae1-249f77b9fd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:57.337640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2c6c0587-1ef3-4e2a-8ae1-249f77b9fd32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:57.354577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:57.448868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c0587-1ef3-4e2a-8ae1-249f77b9fd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:49:57.450388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c0587-1ef3-4e2a-8ae1-249f77b9fd32 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:49:57.468649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 661/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:49:59.772430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c52b5-d05c-4d7b-88d9-76e2e3368ced None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:59.773080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c52b5-d05c-4d7b-88d9-76e2e3368ced None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:59.773080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c52b5-d05c-4d7b-88d9-76e2e3368ced None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:59.774564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 668/1329] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:49:59.780394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90555bdb-a55c-4114-a9f4-454eabc0a00c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:49:59.780821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90555bdb-a55c-4114-a9f4-454eabc0a00c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:49:59.781016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90555bdb-a55c-4114-a9f4-454eabc0a00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:49:59.844114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-90555bdb-a55c-4114-a9f4-454eabc0a00c None None] MFA Rules not processed for user `c987d6e5675d41e7b4123050fafb4d99`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:49:59.845307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-90555bdb-a55c-4114-a9f4-454eabc0a00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:49:59.909065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:49:59.941256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 662/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:49:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:49:59.979600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c52b5-d05c-4d7b-88d9-76e2e3368ced None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:49:59.981120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f38c52b5-d05c-4d7b-88d9-76e2e3368ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:00.086424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c52b5-d05c-4d7b-88d9-76e2e3368ced None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:00.106513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:00.161123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0455530-9d69-495a-b25f-bbb32ad2e280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:00.161770 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455530-9d69-495a-b25f-bbb32ad2e280 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:00.162075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455530-9d69-495a-b25f-bbb32ad2e280 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:00.162348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0455530-9d69-495a-b25f-bbb32ad2e280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:00.163979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a0455530-9d69-495a-b25f-bbb32ad2e280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:00.270738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455530-9d69-495a-b25f-bbb32ad2e280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:00.272189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0455530-9d69-495a-b25f-bbb32ad2e280 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:00.290988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 669/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:49:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:03.036457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763d786-56e6-4432-a684-98edd3b43453 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:03.036988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763d786-56e6-4432-a684-98edd3b43453 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:03.036988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763d786-56e6-4432-a684-98edd3b43453 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:03.038430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 663/1332] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:03.043566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b01ed9-089e-47c3-847c-d4c529cbc14e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:03.044490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b01ed9-089e-47c3-847c-d4c529cbc14e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:03.044672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b01ed9-089e-47c3-847c-d4c529cbc14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:03.109536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:03.112099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-93b01ed9-089e-47c3-847c-d4c529cbc14e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:03.113292 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-93b01ed9-089e-47c3-847c-d4c529cbc14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:03.210203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 670/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:03.245731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763d786-56e6-4432-a684-98edd3b43453 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:03.247419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b763d786-56e6-4432-a684-98edd3b43453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:03.350465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763d786-56e6-4432-a684-98edd3b43453 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:03.424286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837003e-0c93-4d5b-9e4c-08a27ae1bd80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:03.424943 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837003e-0c93-4d5b-9e4c-08a27ae1bd80 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:03.425226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837003e-0c93-4d5b-9e4c-08a27ae1bd80 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:03.425504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837003e-0c93-4d5b-9e4c-08a27ae1bd80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:03.427206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8837003e-0c93-4d5b-9e4c-08a27ae1bd80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:03.456794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:03.541296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837003e-0c93-4d5b-9e4c-08a27ae1bd80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:03.542866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837003e-0c93-4d5b-9e4c-08a27ae1bd80 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:03.569171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 664/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:05.878134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29079e8-2dba-4fde-b604-412b15c32627 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:05.878661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29079e8-2dba-4fde-b604-412b15c32627 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:05.878756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29079e8-2dba-4fde-b604-412b15c32627 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:05.880090 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 671/1335] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:05.885177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b18a7f-1780-45a2-a524-5b7297cead34 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:05.885587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b18a7f-1780-45a2-a524-5b7297cead34 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:05.885724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b18a7f-1780-45a2-a524-5b7297cead34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:05.949020 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-67b18a7f-1780-45a2-a524-5b7297cead34 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:05.950247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-67b18a7f-1780-45a2-a524-5b7297cead34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:06.022439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:06.045222 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 665/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:06.085078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29079e8-2dba-4fde-b604-412b15c32627 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:06.086639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f29079e8-2dba-4fde-b604-412b15c32627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:06.187513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29079e8-2dba-4fde-b604-412b15c32627 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:06.214583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:06.258670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74c990-d9a9-40e8-88d1-91d7490332d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:06.259301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74c990-d9a9-40e8-88d1-91d7490332d9 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:06.259578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74c990-d9a9-40e8-88d1-91d7490332d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:06.259849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74c990-d9a9-40e8-88d1-91d7490332d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:06.261457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ce74c990-d9a9-40e8-88d1-91d7490332d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:06.367044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74c990-d9a9-40e8-88d1-91d7490332d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:06.368495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74c990-d9a9-40e8-88d1-91d7490332d9 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:06.388306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 672/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:09.255223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2941-c26f-4d75-870e-7ee9beac329f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:09.255783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2941-c26f-4d75-870e-7ee9beac329f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:09.255783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e2941-c26f-4d75-870e-7ee9beac329f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:09.257156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 666/1338] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:09.262888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82f4e1-0586-4756-8b57-6ebdfa75bf1c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:09.263267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82f4e1-0586-4756-8b57-6ebdfa75bf1c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:09.263461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82f4e1-0586-4756-8b57-6ebdfa75bf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:09.329085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1a82f4e1-0586-4756-8b57-6ebdfa75bf1c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:09.330334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1a82f4e1-0586-4756-8b57-6ebdfa75bf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:09.338377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:09.425774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 673/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:09.460883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e2941-c26f-4d75-870e-7ee9beac329f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:09.463926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-565e2941-c26f-4d75-870e-7ee9beac329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:09.571455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e2941-c26f-4d75-870e-7ee9beac329f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:09.627447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca631c2-76e2-4ec1-84d0-a1ff58f0c25e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:09.628090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca631c2-76e2-4ec1-84d0-a1ff58f0c25e service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:09.628379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca631c2-76e2-4ec1-84d0-a1ff58f0c25e service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:09.628659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca631c2-76e2-4ec1-84d0-a1ff58f0c25e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:09.630299 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9ca631c2-76e2-4ec1-84d0-a1ff58f0c25e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:09.685260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:09.732411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca631c2-76e2-4ec1-84d0-a1ff58f0c25e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:09.733850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca631c2-76e2-4ec1-84d0-a1ff58f0c25e service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:09.753163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 667/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:14.190030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9e2b2-0a49-451d-b499-7ac6ae360451 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:14.190606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9e2b2-0a49-451d-b499-7ac6ae360451 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:14.190606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9e2b2-0a49-451d-b499-7ac6ae360451 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:14.192207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 674/1341] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:14.197414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3a62a-8c26-474d-8816-ea1b288d4fbb None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:14.197863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3a62a-8c26-474d-8816-ea1b288d4fbb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:14.197998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3a62a-8c26-474d-8816-ea1b288d4fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:14.244145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8aa3a62a-8c26-474d-8816-ea1b288d4fbb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:14.245275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8aa3a62a-8c26-474d-8816-ea1b288d4fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:14.323326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 668/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:14.361396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9e2b2-0a49-451d-b499-7ac6ae360451 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:14.363537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8da9e2b2-0a49-451d-b499-7ac6ae360451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:14.386384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9e2b2-0a49-451d-b499-7ac6ae360451 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:14.389874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ab411-de6d-4b01-bb9f-2ab24d7288ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:14.390530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ab411-de6d-4b01-bb9f-2ab24d7288ef service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:14.390818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ab411-de6d-4b01-bb9f-2ab24d7288ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:14.391097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ab411-de6d-4b01-bb9f-2ab24d7288ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:14.400188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ab411-de6d-4b01-bb9f-2ab24d7288ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:14.401606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ab411-de6d-4b01-bb9f-2ab24d7288ef service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:14.410975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 675/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:17.805641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c3ebf-18d2-4a8f-a44c-2350fc11e9f6 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:17.806193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c3ebf-18d2-4a8f-a44c-2350fc11e9f6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:17.806307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c3ebf-18d2-4a8f-a44c-2350fc11e9f6 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:17.807603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 669/1344] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:17.812878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e96c4c-6d69-4819-b4a1-9596fa865a91 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:17.813317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e96c4c-6d69-4819-b4a1-9596fa865a91 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:17.813413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e96c4c-6d69-4819-b4a1-9596fa865a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:17.841675 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d0e96c4c-6d69-4819-b4a1-9596fa865a91 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:17.842907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d0e96c4c-6d69-4819-b4a1-9596fa865a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:17.850698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 676/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:17.888261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c3ebf-18d2-4a8f-a44c-2350fc11e9f6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:17.898889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c3ebf-18d2-4a8f-a44c-2350fc11e9f6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:17.902758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd4393-f7b7-4f0e-91c4-7beb5b702c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:17.903396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd4393-f7b7-4f0e-91c4-7beb5b702c0f service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:17.903687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd4393-f7b7-4f0e-91c4-7beb5b702c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:17.903967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd4393-f7b7-4f0e-91c4-7beb5b702c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:17.912922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd4393-f7b7-4f0e-91c4-7beb5b702c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:17.914392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd4393-f7b7-4f0e-91c4-7beb5b702c0f service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:17.924285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 670/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:22.469711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da410b9e-3993-47c6-abe4-de85706296bf None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:22.470307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da410b9e-3993-47c6-abe4-de85706296bf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:22.470307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da410b9e-3993-47c6-abe4-de85706296bf None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:22.471656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 677/1347] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:22.476801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b410-e0c8-461c-9773-114bb77a91f6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:22.477195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b410-e0c8-461c-9773-114bb77a91f6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:22.477362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5b410-e0c8-461c-9773-114bb77a91f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:22.495473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-48a5b410-e0c8-461c-9773-114bb77a91f6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:22.497221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-48a5b410-e0c8-461c-9773-114bb77a91f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:22.505198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 671/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:22.544696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da410b9e-3993-47c6-abe4-de85706296bf None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:22.555396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da410b9e-3993-47c6-abe4-de85706296bf None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:22.558877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960fd19a-cb55-4fdb-b36c-b6276a487d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:22.559515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fd19a-cb55-4fdb-b36c-b6276a487d4c service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:22.559800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fd19a-cb55-4fdb-b36c-b6276a487d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:22.560077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960fd19a-cb55-4fdb-b36c-b6276a487d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:22.569009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fd19a-cb55-4fdb-b36c-b6276a487d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:22.570460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960fd19a-cb55-4fdb-b36c-b6276a487d4c service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:22.579950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 678/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:27.915049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6f8c1-0eef-4418-a430-ddac2f9111e6 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:27.915812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6f8c1-0eef-4418-a430-ddac2f9111e6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:27.916240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6f8c1-0eef-4418-a430-ddac2f9111e6 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:27.918634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 672/1350] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 09:50:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:27.923696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a60832-7615-483f-9165-d427c1bfbd8d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:27.924464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a60832-7615-483f-9165-d427c1bfbd8d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:27.924915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a60832-7615-483f-9165-d427c1bfbd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:28.017380 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-98a60832-7615-483f-9165-d427c1bfbd8d None None] MFA Rules not processed for user `566c0c4bdcaf4c13b060af4bbff5bc04`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:28.018904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-98a60832-7615-483f-9165-d427c1bfbd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:28.079029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:28.160682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 679/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 09:50:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:32.465002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b50df-b417-4412-91bc-5c9a745bdc45 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:32.465864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b50df-b417-4412-91bc-5c9a745bdc45 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:32.466164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b50df-b417-4412-91bc-5c9a745bdc45 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:32.467526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 673/1352] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 09:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:32.473408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7fbcf-e3d1-44b4-b9fe-9876f74a21c7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:32.474216 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7fbcf-e3d1-44b4-b9fe-9876f74a21c7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:32.474571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7fbcf-e3d1-44b4-b9fe-9876f74a21c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:32.490071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=80 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:32.492909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-eae7fbcf-e3d1-44b4-b9fe-9876f74a21c7 None None] MFA Rules not processed for user `566c0c4bdcaf4c13b060af4bbff5bc04`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:32.494225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-eae7fbcf-e3d1-44b4-b9fe-9876f74a21c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:32.503903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 680/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 09:50:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:34.217241 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03410ba-1087-4930-b23c-1ac0e3c6823c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:34.217826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03410ba-1087-4930-b23c-1ac0e3c6823c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:34.217826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03410ba-1087-4930-b23c-1ac0e3c6823c None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:34.219231 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 674/1354] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 09:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:34.224838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37eda8-0064-4789-a6fe-ddc9567c7f94 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:34.225357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37eda8-0064-4789-a6fe-ddc9567c7f94 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:34.225522 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37eda8-0064-4789-a6fe-ddc9567c7f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:34.244901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6b37eda8-0064-4789-a6fe-ddc9567c7f94 None None] MFA Rules not processed for user `566c0c4bdcaf4c13b060af4bbff5bc04`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:34.246739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6b37eda8-0064-4789-a6fe-ddc9567c7f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:34.255205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 681/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 09:50:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:37.980662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff7a17-7636-43e2-a92a-e502eaceefe8 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:37.981216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff7a17-7636-43e2-a92a-e502eaceefe8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:37.981216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff7a17-7636-43e2-a92a-e502eaceefe8 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:37.982476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 675/1356] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:37.987842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae5c0-eda4-4e3f-bd62-3d1541378dc0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:37.988270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae5c0-eda4-4e3f-bd62-3d1541378dc0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:37.988402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ae5c0-eda4-4e3f-bd62-3d1541378dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:38.013318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c41ae5c0-eda4-4e3f-bd62-3d1541378dc0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:38.014566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c41ae5c0-eda4-4e3f-bd62-3d1541378dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:38.023058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 682/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:38.079790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9358-c3bc-4c5e-92fc-095233809158 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:38.080075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9358-c3bc-4c5e-92fc-095233809158 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:38.080710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e9358-c3bc-4c5e-92fc-095233809158 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:38.082356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 676/1358] 10.222.0.42 () {54 vars in 961 bytes} [Fri Jun 3 09:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:38.097777 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639894c-1ed8-474c-b7ee-d14351bef80d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:38.100229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639894c-1ed8-474c-b7ee-d14351bef80d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:38.101256 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639894c-1ed8-474c-b7ee-d14351bef80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:38.166685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9639894c-1ed8-474c-b7ee-d14351bef80d None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:38.169332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9639894c-1ed8-474c-b7ee-d14351bef80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:38.266672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 683/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Fri Jun 3 09:50:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:38.271754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9358-c3bc-4c5e-92fc-095233809158 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:38.287123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e9358-c3bc-4c5e-92fc-095233809158 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:38.292863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a962f-6239-4c97-a553-538ea3560ca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:38.293750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a962f-6239-4c97-a553-538ea3560ca6 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:38.294303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a962f-6239-4c97-a553-538ea3560ca6 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:38.294777 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a962f-6239-4c97-a553-538ea3560ca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:38.304084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a962f-6239-4c97-a553-538ea3560ca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:38.305939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a962f-6239-4c97-a553-538ea3560ca6 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:38.316952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 677/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:40.764757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f757c-fe81-4b57-aea7-62bf0dbbcd88 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:40.765270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f757c-fe81-4b57-aea7-62bf0dbbcd88 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:40.765382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f757c-fe81-4b57-aea7-62bf0dbbcd88 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:40.766865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 684/1361] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:40.771956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a81baa-9a42-4207-8f45-2d257e744256 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:40.772370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a81baa-9a42-4207-8f45-2d257e744256 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:40.772506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a81baa-9a42-4207-8f45-2d257e744256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:40.790473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-28a81baa-9a42-4207-8f45-2d257e744256 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:40.791749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-28a81baa-9a42-4207-8f45-2d257e744256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:40.799533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 678/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:40.843616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f757c-fe81-4b57-aea7-62bf0dbbcd88 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:40.859914 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f757c-fe81-4b57-aea7-62bf0dbbcd88 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:40.868408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8de54-32d1-4f78-bf11-3a500c59bab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:40.869065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8de54-32d1-4f78-bf11-3a500c59bab5 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:40.869348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8de54-32d1-4f78-bf11-3a500c59bab5 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:40.869625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8de54-32d1-4f78-bf11-3a500c59bab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:40.879469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8de54-32d1-4f78-bf11-3a500c59bab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:40.881050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8de54-32d1-4f78-bf11-3a500c59bab5 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:40.892073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 685/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:43.568344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf786-3005-43e3-9055-71b4072f3276 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:43.568826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf786-3005-43e3-9055-71b4072f3276 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:43.568927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf786-3005-43e3-9055-71b4072f3276 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:43.570293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 679/1364] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:43.576092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f3e16-94e5-45f0-addc-e91c70b24059 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:43.576975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f3e16-94e5-45f0-addc-e91c70b24059 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:43.577420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f3e16-94e5-45f0-addc-e91c70b24059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:43.596548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a19f3e16-94e5-45f0-addc-e91c70b24059 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:43.597805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a19f3e16-94e5-45f0-addc-e91c70b24059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:43.605635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 686/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:43.652559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75a0d7-431f-4b21-b68f-8058653ebd8f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:43.653455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75a0d7-431f-4b21-b68f-8058653ebd8f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:43.653839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75a0d7-431f-4b21-b68f-8058653ebd8f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:43.655483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 680/1366] 10.222.0.42 () {54 vars in 961 bytes} [Fri Jun 3 09:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:43.660795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5040c2-7e0b-4707-92c4-26f73640db66 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:43.661085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5040c2-7e0b-4707-92c4-26f73640db66 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:43.661365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5040c2-7e0b-4707-92c4-26f73640db66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:43.679566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0c5040c2-7e0b-4707-92c4-26f73640db66 None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:43.680721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0c5040c2-7e0b-4707-92c4-26f73640db66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:43.690734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 687/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Fri Jun 3 09:50:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:43.698729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75a0d7-431f-4b21-b68f-8058653ebd8f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:43.713072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75a0d7-431f-4b21-b68f-8058653ebd8f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:43.719543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ccdc43-02fa-43cf-99e6-9a6eda95a271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:43.720181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccdc43-02fa-43cf-99e6-9a6eda95a271 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:43.720459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccdc43-02fa-43cf-99e6-9a6eda95a271 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:43.720731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ccdc43-02fa-43cf-99e6-9a6eda95a271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:43.729253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccdc43-02fa-43cf-99e6-9a6eda95a271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:43.731471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ccdc43-02fa-43cf-99e6-9a6eda95a271 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:43.740690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 681/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:50:46.246176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd6ef5-2d62-431d-b6cd-3b12c63a62c1 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:46.246738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd6ef5-2d62-431d-b6cd-3b12c63a62c1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:46.247129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd6ef5-2d62-431d-b6cd-3b12c63a62c1 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:46.249042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 688/1369] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:46.254038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cae175-df84-4c05-a992-0c11dec855e5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:46.254490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cae175-df84-4c05-a992-0c11dec855e5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:46.254621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cae175-df84-4c05-a992-0c11dec855e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:46.274028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e4cae175-df84-4c05-a992-0c11dec855e5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:46.275706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e4cae175-df84-4c05-a992-0c11dec855e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:46.284234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 682/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:46.331909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd6ef5-2d62-431d-b6cd-3b12c63a62c1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:46.347224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd6ef5-2d62-431d-b6cd-3b12c63a62c1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:46.352949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0f746-1983-4d3d-9aaf-621831cf6b91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:46.356807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0f746-1983-4d3d-9aaf-621831cf6b91 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:46.356807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0f746-1983-4d3d-9aaf-621831cf6b91 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:46.356807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0f746-1983-4d3d-9aaf-621831cf6b91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:46.362452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e35447-e9c7-4fbb-9bc8-2e1dd431c5df None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:46.362740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e35447-e9c7-4fbb-9bc8-2e1dd431c5df None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:46.363069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e35447-e9c7-4fbb-9bc8-2e1dd431c5df None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:46.364311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0f746-1983-4d3d-9aaf-621831cf6b91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:46.364897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 683/1371] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:46.365985 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0f746-1983-4d3d-9aaf-621831cf6b91 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:46.370071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc40a3-5b1c-4e8b-b77a-c1e791b7d8e5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:46.370980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc40a3-5b1c-4e8b-b77a-c1e791b7d8e5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:46.371326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc40a3-5b1c-4e8b-b77a-c1e791b7d8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:46.377333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 689/1372] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:46.390176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-28bc40a3-5b1c-4e8b-b77a-c1e791b7d8e5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:46.391867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-28bc40a3-5b1c-4e8b-b77a-c1e791b7d8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:46.399686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 684/1373] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:50:46.708533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4d7d3-97c6-4976-a806-d17a19a3caec None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:46.708533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4d7d3-97c6-4976-a806-d17a19a3caec None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:46.708725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4d7d3-97c6-4976-a806-d17a19a3caec None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:46.710078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 690/1374] 10.222.0.42 () {54 vars in 963 bytes} [Fri Jun 3 09:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:46.715663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc326ac-f9ba-4ae8-9f9f-4b9ecb6ba02b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:46.715954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc326ac-f9ba-4ae8-9f9f-4b9ecb6ba02b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:46.716233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc326ac-f9ba-4ae8-9f9f-4b9ecb6ba02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:46.777484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8fc326ac-f9ba-4ae8-9f9f-4b9ecb6ba02b None None] MFA Rules not processed for user `aee48298b3834984a12fe2fe0a6f9a77`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:46.778868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8fc326ac-f9ba-4ae8-9f9f-4b9ecb6ba02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:46.849404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 685/1375] 10.222.0.42 () {58 vars in 1060 bytes} [Fri Jun 3 09:50:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:46.857200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4d7d3-97c6-4976-a806-d17a19a3caec None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:46.869103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4d7d3-97c6-4976-a806-d17a19a3caec None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:46.873624 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b784b1-aefe-4201-9268-bce13fe5295b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:46.874644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b784b1-aefe-4201-9268-bce13fe5295b service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:46.874996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b784b1-aefe-4201-9268-bce13fe5295b service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:46.875400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b784b1-aefe-4201-9268-bce13fe5295b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:46.886016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b784b1-aefe-4201-9268-bce13fe5295b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:46.887568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b784b1-aefe-4201-9268-bce13fe5295b service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:46.899608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 691/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:48.950620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c1547-42fc-46cf-8225-06777d5047a5 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:48.951194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c1547-42fc-46cf-8225-06777d5047a5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:48.951194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c1547-42fc-46cf-8225-06777d5047a5 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:48.952562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 686/1377] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:48.958164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be79e0d-7012-4786-abb0-712e5a3858bc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:48.958602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be79e0d-7012-4786-abb0-712e5a3858bc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:48.958711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be79e0d-7012-4786-abb0-712e5a3858bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:48.977438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1be79e0d-7012-4786-abb0-712e5a3858bc None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:48.978735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1be79e0d-7012-4786-abb0-712e5a3858bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:48.987177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 692/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:49.031118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c1547-42fc-46cf-8225-06777d5047a5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:49.046325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c1547-42fc-46cf-8225-06777d5047a5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:49.051897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa10da-ba1a-4508-bcfe-4055af6a34fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:49.052699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa10da-ba1a-4508-bcfe-4055af6a34fc service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:49.052984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa10da-ba1a-4508-bcfe-4055af6a34fc service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:49.053258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa10da-ba1a-4508-bcfe-4055af6a34fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:49.061669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa10da-ba1a-4508-bcfe-4055af6a34fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:49.063310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa10da-ba1a-4508-bcfe-4055af6a34fc service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:49.072825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 687/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:51.466959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367e651-9ae2-4d50-8225-196e02ec5b0f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:51.467494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367e651-9ae2-4d50-8225-196e02ec5b0f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:51.467614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367e651-9ae2-4d50-8225-196e02ec5b0f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:51.469112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 693/1380] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:51.474223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7771c5-ff82-4f6b-b495-db858f02173b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:51.474700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7771c5-ff82-4f6b-b495-db858f02173b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:51.474805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7771c5-ff82-4f6b-b495-db858f02173b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:51.492214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9d7771c5-ff82-4f6b-b495-db858f02173b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:51.493373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9d7771c5-ff82-4f6b-b495-db858f02173b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:51.501756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 688/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:51.545784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367e651-9ae2-4d50-8225-196e02ec5b0f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:51.554112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:50:51.561374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367e651-9ae2-4d50-8225-196e02ec5b0f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:51.568399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bacdca-60db-4ddd-a672-763d3855f01f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:51.569036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bacdca-60db-4ddd-a672-763d3855f01f service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:51.569326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bacdca-60db-4ddd-a672-763d3855f01f service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:51.569606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bacdca-60db-4ddd-a672-763d3855f01f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:51.578570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bacdca-60db-4ddd-a672-763d3855f01f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:51.582914 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bacdca-60db-4ddd-a672-763d3855f01f service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:51.590349 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 694/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:54.017518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa3628-e89d-436c-93ae-5baeaa64e222 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:54.018073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa3628-e89d-436c-93ae-5baeaa64e222 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:54.018073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa3628-e89d-436c-93ae-5baeaa64e222 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:54.019655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 689/1383] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:54.025004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094084b-fafc-4a9f-b1bf-3a397df19498 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:54.025441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094084b-fafc-4a9f-b1bf-3a397df19498 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:54.025539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094084b-fafc-4a9f-b1bf-3a397df19498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:54.043682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6094084b-fafc-4a9f-b1bf-3a397df19498 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:54.044890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6094084b-fafc-4a9f-b1bf-3a397df19498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:54.052927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 695/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:54.094176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa3628-e89d-436c-93ae-5baeaa64e222 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:54.110328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa3628-e89d-436c-93ae-5baeaa64e222 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:54.116104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5754e56-3111-4d25-82a8-256c2dbed04e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:54.116760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5754e56-3111-4d25-82a8-256c2dbed04e service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:54.117432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5754e56-3111-4d25-82a8-256c2dbed04e service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:54.117758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5754e56-3111-4d25-82a8-256c2dbed04e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:54.127779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5754e56-3111-4d25-82a8-256c2dbed04e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:54.129522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5754e56-3111-4d25-82a8-256c2dbed04e service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:54.139914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 690/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:56.383255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ce0f1-f887-4fcc-9d6e-09eb1c3c96e6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:56.383849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ce0f1-f887-4fcc-9d6e-09eb1c3c96e6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:56.384050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ce0f1-f887-4fcc-9d6e-09eb1c3c96e6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:56.385873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 696/1386] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:56.391066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674c66f-ea52-4c8b-8c9f-ad1e0726efab None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:56.391516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674c66f-ea52-4c8b-8c9f-ad1e0726efab None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:56.391624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674c66f-ea52-4c8b-8c9f-ad1e0726efab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:56.409610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1674c66f-ea52-4c8b-8c9f-ad1e0726efab None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:56.410841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1674c66f-ea52-4c8b-8c9f-ad1e0726efab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:56.419350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 691/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:50:56.464717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ce0f1-f887-4fcc-9d6e-09eb1c3c96e6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:56.480271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ce0f1-f887-4fcc-9d6e-09eb1c3c96e6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:56.485957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d6934-4a00-4742-8dcf-ef8a363de146 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:56.486634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6934-4a00-4742-8dcf-ef8a363de146 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:56.486919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6934-4a00-4742-8dcf-ef8a363de146 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:56.487211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6934-4a00-4742-8dcf-ef8a363de146 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:56.495908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d6934-4a00-4742-8dcf-ef8a363de146 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:56.497561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d6934-4a00-4742-8dcf-ef8a363de146 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:56.507623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 697/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:50:59.223135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b6279-d1d7-4fd5-bcd0-47052a083036 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:59.223695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b6279-d1d7-4fd5-bcd0-47052a083036 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:59.223695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b6279-d1d7-4fd5-bcd0-47052a083036 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:59.225062 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 692/1389] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:50:59.231107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ca384-38d6-41ff-907a-183ca32832ec None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:59.231679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ca384-38d6-41ff-907a-183ca32832ec None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:59.231788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ca384-38d6-41ff-907a-183ca32832ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:59.249484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-048ca384-38d6-41ff-907a-183ca32832ec None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:50:59.250768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-048ca384-38d6-41ff-907a-183ca32832ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:50:59.259026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 698/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:50:59.305508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b6279-d1d7-4fd5-bcd0-47052a083036 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:50:59.324096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b6279-d1d7-4fd5-bcd0-47052a083036 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:50:59.331375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2332fce-84a9-4694-96ef-c17e907eddc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:50:59.332254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2332fce-84a9-4694-96ef-c17e907eddc8 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:50:59.332621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2332fce-84a9-4694-96ef-c17e907eddc8 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:50:59.332996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2332fce-84a9-4694-96ef-c17e907eddc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:50:59.343516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2332fce-84a9-4694-96ef-c17e907eddc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:50:59.345588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2332fce-84a9-4694-96ef-c17e907eddc8 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:50:59.358279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 693/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:50:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:01.734254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1f7e5-8162-4b4b-b691-db69603f4ab8 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:01.734763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1f7e5-8162-4b4b-b691-db69603f4ab8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:01.734871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1f7e5-8162-4b4b-b691-db69603f4ab8 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:01.736183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 699/1392] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:01.741438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cfd13-bc04-40b4-80f5-bb5dd9639ff0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:01.741908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cfd13-bc04-40b4-80f5-bb5dd9639ff0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:01.742020 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391cfd13-bc04-40b4-80f5-bb5dd9639ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:01.760239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-391cfd13-bc04-40b4-80f5-bb5dd9639ff0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:01.761435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-391cfd13-bc04-40b4-80f5-bb5dd9639ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:01.769246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 694/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:01.813574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1f7e5-8162-4b4b-b691-db69603f4ab8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:01.827647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1f7e5-8162-4b4b-b691-db69603f4ab8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:01.832889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deae47d-210b-4cee-9dbc-b15719e6fe7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:01.833512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deae47d-210b-4cee-9dbc-b15719e6fe7e service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:01.833796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deae47d-210b-4cee-9dbc-b15719e6fe7e service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:01.834105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deae47d-210b-4cee-9dbc-b15719e6fe7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:01.842912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deae47d-210b-4cee-9dbc-b15719e6fe7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:01.844501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deae47d-210b-4cee-9dbc-b15719e6fe7e service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:01.854935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 700/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:03.197713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2fafe-f276-46c2-b512-15bfef41e5b0 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:03.198325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2fafe-f276-46c2-b512-15bfef41e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:03.198325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2fafe-f276-46c2-b512-15bfef41e5b0 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:03.199675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 695/1395] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:03.204956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75321c7-1d28-49b2-bcdb-c1955619593a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:03.205443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75321c7-1d28-49b2-bcdb-c1955619593a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:03.205594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75321c7-1d28-49b2-bcdb-c1955619593a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:03.223222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c75321c7-1d28-49b2-bcdb-c1955619593a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:03.224447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c75321c7-1d28-49b2-bcdb-c1955619593a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:03.232799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 701/1396] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:03.288448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325029-9f5b-4efd-b900-7e9f3be4806f None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:03.288742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325029-9f5b-4efd-b900-7e9f3be4806f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:03.289039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325029-9f5b-4efd-b900-7e9f3be4806f None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:03.290432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 696/1397] 10.222.0.42 () {54 vars in 962 bytes} [Fri Jun 3 09:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:03.296051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447fcb8-7577-4baf-8ed4-2c715060d86a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:03.296519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447fcb8-7577-4baf-8ed4-2c715060d86a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:03.296967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447fcb8-7577-4baf-8ed4-2c715060d86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:03.344196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2447fcb8-7577-4baf-8ed4-2c715060d86a None None] MFA Rules not processed for user `d59a351971e94aa38a22e8113207e5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:03.345369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2447fcb8-7577-4baf-8ed4-2c715060d86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:03.405242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 702/1398] 10.222.0.42 () {58 vars in 1059 bytes} [Fri Jun 3 09:51:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:51:03.412537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b325029-9f5b-4efd-b900-7e9f3be4806f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:03.418530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=87 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:51:03.424084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b325029-9f5b-4efd-b900-7e9f3be4806f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:03.427628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2f684-5229-4506-a677-3720d1a48486 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd59a351971e94aa38a22e8113207e5b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:03.428252 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2f684-5229-4506-a677-3720d1a48486 service glance] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:03.428535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2f684-5229-4506-a677-3720d1a48486 service glance] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:03.428811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2f684-5229-4506-a677-3720d1a48486 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:03.437978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2f684-5229-4506-a677-3720d1a48486 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:03.439478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2f684-5229-4506-a677-3720d1a48486 service glance] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:03.449052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 697/1399] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:04.073832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648f6b0-5428-4794-8e80-469f02d79bc9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:04.074154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648f6b0-5428-4794-8e80-469f02d79bc9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:04.074436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648f6b0-5428-4794-8e80-469f02d79bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:04.121578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0648f6b0-5428-4794-8e80-469f02d79bc9 None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:04.122807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0648f6b0-5428-4794-8e80-469f02d79bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:04.171342 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:51:04.188955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 703/1400] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 09:51:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:51:04.234891 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50b1b-9f49-4c31-b3cc-021965388113 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:04.235382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50b1b-9f49-4c31-b3cc-021965388113 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:04.235607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50b1b-9f49-4c31-b3cc-021965388113 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:04.237086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 698/1401] 10.222.0.42 () {54 vars in 942 bytes} [Fri Jun 3 09:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:04.242783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babbeca-99d5-42d5-8446-c568980a882e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:04.243255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babbeca-99d5-42d5-8446-c568980a882e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:04.243360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babbeca-99d5-42d5-8446-c568980a882e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:04.243504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f25a10-e5d6-410e-8b87-cc96fe3a6296 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:04.243848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f25a10-e5d6-410e-8b87-cc96fe3a6296 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:04.244162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f25a10-e5d6-410e-8b87-cc96fe3a6296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:04.245270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 704/1402] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:04.250394 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de3431-7e9b-4ee2-9cfd-296bf7f07894 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:04.250684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de3431-7e9b-4ee2-9cfd-296bf7f07894 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:04.250973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de3431-7e9b-4ee2-9cfd-296bf7f07894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:04.269441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e8de3431-7e9b-4ee2-9cfd-296bf7f07894 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:04.274547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e8de3431-7e9b-4ee2-9cfd-296bf7f07894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:04.279111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 705/1403] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:51:04.293944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-54f25a10-e5d6-410e-8b87-cc96fe3a6296 None None] MFA Rules not processed for user `37186dc0010247cd936f160b5eb0298b`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:04.300289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-54f25a10-e5d6-410e-8b87-cc96fe3a6296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:04.323611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de3431-7e9b-4ee2-9cfd-296bf7f07894 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:04.336026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de3431-7e9b-4ee2-9cfd-296bf7f07894 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:04.341991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad90f3-b0c2-4beb-b60b-bae3238b12e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:04.342642 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad90f3-b0c2-4beb-b60b-bae3238b12e3 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:04.342927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad90f3-b0c2-4beb-b60b-bae3238b12e3 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:04.343210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad90f3-b0c2-4beb-b60b-bae3238b12e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:04.351985 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad90f3-b0c2-4beb-b60b-bae3238b12e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:04.353583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad90f3-b0c2-4beb-b60b-bae3238b12e3 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:04.363327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 706/1404] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:51:04.369290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 699/1405] 10.222.0.42 () {58 vars in 1039 bytes} [Fri Jun 3 09:51:04 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:04.377595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad90f3-b0c2-4beb-b60b-bae3238b12e3 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:04.389090 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad90f3-b0c2-4beb-b60b-bae3238b12e3 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:04.394097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777044b2-fd7e-4bba-a275-0f7f7473f2d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:04.394770 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777044b2-fd7e-4bba-a275-0f7f7473f2d8 service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:04.395053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777044b2-fd7e-4bba-a275-0f7f7473f2d8 service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:04.395329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777044b2-fd7e-4bba-a275-0f7f7473f2d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:04.403703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777044b2-fd7e-4bba-a275-0f7f7473f2d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:04.405565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777044b2-fd7e-4bba-a275-0f7f7473f2d8 service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:04.413745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 707/1406] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 09:51:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 09:51:06.769925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0044662-fc11-4560-b610-58a27d7d027d None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:06.770487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0044662-fc11-4560-b610-58a27d7d027d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:06.770487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0044662-fc11-4560-b610-58a27d7d027d None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:06.771939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 700/1407] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:06.778230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff9a20-e23f-4017-8f4c-1f894bc96d13 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:06.779153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff9a20-e23f-4017-8f4c-1f894bc96d13 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:06.779437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff9a20-e23f-4017-8f4c-1f894bc96d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:06.797395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-11ff9a20-e23f-4017-8f4c-1f894bc96d13 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:06.798617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-11ff9a20-e23f-4017-8f4c-1f894bc96d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:06.813974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 708/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:06.852846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0044662-fc11-4560-b610-58a27d7d027d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:06.871851 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0044662-fc11-4560-b610-58a27d7d027d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:06.885013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55520f-6ed3-446c-bfb1-a298fc7ce3b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:06.885806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55520f-6ed3-446c-bfb1-a298fc7ce3b6 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:06.886197 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55520f-6ed3-446c-bfb1-a298fc7ce3b6 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:06.886538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55520f-6ed3-446c-bfb1-a298fc7ce3b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:06.896216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55520f-6ed3-446c-bfb1-a298fc7ce3b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:06.897843 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55520f-6ed3-446c-bfb1-a298fc7ce3b6 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:06.910040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 701/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:16.862825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=22 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:51:16.903895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=55 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:51:29.320581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794298a8-7f4f-4c53-a808-8cff4f5a911d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:29.321235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794298a8-7f4f-4c53-a808-8cff4f5a911d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:29.321235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794298a8-7f4f-4c53-a808-8cff4f5a911d None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:29.323450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 709/1410] 10.222.0.42 () {54 vars in 918 bytes} [Fri Jun 3 09:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:29.328509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f49d34-8c83-45bb-861b-3d07351509fa None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:29.328944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f49d34-8c83-45bb-861b-3d07351509fa None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:29.329054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f49d34-8c83-45bb-861b-3d07351509fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:29.346890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d4f49d34-8c83-45bb-861b-3d07351509fa None None] MFA Rules not processed for user `566c0c4bdcaf4c13b060af4bbff5bc04`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:29.348078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d4f49d34-8c83-45bb-861b-3d07351509fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:29.356752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 702/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Fri Jun 3 09:51:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:29.371154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a61fc-1928-415b-aef4-d447347801a2 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:29.371440 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a61fc-1928-415b-aef4-d447347801a2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:29.371717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a61fc-1928-415b-aef4-d447347801a2 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:29.373026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 710/1412] 10.222.0.42 () {54 vars in 918 bytes} [Fri Jun 3 09:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:29.377506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5f1fa-ad15-4b53-8979-d34d30db7fbb None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:29.377792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5f1fa-ad15-4b53-8979-d34d30db7fbb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:29.378101 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5f1fa-ad15-4b53-8979-d34d30db7fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:29.395649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-30f5f1fa-ad15-4b53-8979-d34d30db7fbb None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:29.396825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-30f5f1fa-ad15-4b53-8979-d34d30db7fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:29.406034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 703/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Fri Jun 3 09:51:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:32.120858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303be257-e914-42b1-969e-0a4777eb993e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:32.121411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303be257-e914-42b1-969e-0a4777eb993e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:32.121411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303be257-e914-42b1-969e-0a4777eb993e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:32.122813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:32.128146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862ca6-c71b-4ff0-80f0-828e4c1f170e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:32.128600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862ca6-c71b-4ff0-80f0-828e4c1f170e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:32.128710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da862ca6-c71b-4ff0-80f0-828e4c1f170e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:32.146056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-da862ca6-c71b-4ff0-80f0-828e4c1f170e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:32.147226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-da862ca6-c71b-4ff0-80f0-828e4c1f170e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:32.155571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:32.198211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303be257-e914-42b1-969e-0a4777eb993e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:32.212626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303be257-e914-42b1-969e-0a4777eb993e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:32.217815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0edf36-ddf1-4fea-aa1a-f54dee5b8d6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:32.218470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0edf36-ddf1-4fea-aa1a-f54dee5b8d6e service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:32.218750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0edf36-ddf1-4fea-aa1a-f54dee5b8d6e service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:32.219026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0edf36-ddf1-4fea-aa1a-f54dee5b8d6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:32.226953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0edf36-ddf1-4fea-aa1a-f54dee5b8d6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:32.229201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0edf36-ddf1-4fea-aa1a-f54dee5b8d6e service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:32.238388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 712/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:33.209942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc0b54-e85d-4cef-9ad0-f9bd108dc21e None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:33.210511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc0b54-e85d-4cef-9ad0-f9bd108dc21e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:33.210511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc0b54-e85d-4cef-9ad0-f9bd108dc21e None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:33.211874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:33.216886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37c429-5ba0-49b3-900b-0f57acffc7d8 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:33.217333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37c429-5ba0-49b3-900b-0f57acffc7d8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:33.217432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37c429-5ba0-49b3-900b-0f57acffc7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:33.235921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3e37c429-5ba0-49b3-900b-0f57acffc7d8 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:33.237161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3e37c429-5ba0-49b3-900b-0f57acffc7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:33.245749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:33.414879 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc0b54-e85d-4cef-9ad0-f9bd108dc21e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:33.429513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc0b54-e85d-4cef-9ad0-f9bd108dc21e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:33.434892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ee664-bb42-42d8-891d-acbad17e5060 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:33.435527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ee664-bb42-42d8-891d-acbad17e5060 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:33.435809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ee664-bb42-42d8-891d-acbad17e5060 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:33.436087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ee664-bb42-42d8-891d-acbad17e5060 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:33.444033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ee664-bb42-42d8-891d-acbad17e5060 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:33.445649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ee664-bb42-42d8-891d-acbad17e5060 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:33.456571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 706/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:34.394655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba47fd-798b-4b74-931a-ec1824c92ab7 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:34.395249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba47fd-798b-4b74-931a-ec1824c92ab7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:34.395249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba47fd-798b-4b74-931a-ec1824c92ab7 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:34.396577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:34.402291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7916753-480f-4ffb-87c7-98e5d1627f74 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:34.402702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7916753-480f-4ffb-87c7-98e5d1627f74 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:34.402849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7916753-480f-4ffb-87c7-98e5d1627f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:34.421513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a7916753-480f-4ffb-87c7-98e5d1627f74 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:34.422734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a7916753-480f-4ffb-87c7-98e5d1627f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:34.430465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:34.578853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba47fd-798b-4b74-931a-ec1824c92ab7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:34.593300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba47fd-798b-4b74-931a-ec1824c92ab7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:34.598613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98389ef-3911-4001-b25a-44e5ca09766b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:34.599757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98389ef-3911-4001-b25a-44e5ca09766b service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:34.599757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98389ef-3911-4001-b25a-44e5ca09766b service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:34.599757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98389ef-3911-4001-b25a-44e5ca09766b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:34.607688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98389ef-3911-4001-b25a-44e5ca09766b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:34.609262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98389ef-3911-4001-b25a-44e5ca09766b service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:34.618993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 715/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:35.520075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b487d-76f4-4b2f-9636-b2930ef95716 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:35.520721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b487d-76f4-4b2f-9636-b2930ef95716 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:35.520721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b487d-76f4-4b2f-9636-b2930ef95716 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:35.522026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:35.527206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964e6c8-1999-4aac-b7da-abf2a0e91094 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:35.527619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964e6c8-1999-4aac-b7da-abf2a0e91094 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:35.527763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964e6c8-1999-4aac-b7da-abf2a0e91094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:35.546083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4964e6c8-1999-4aac-b7da-abf2a0e91094 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:35.547294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4964e6c8-1999-4aac-b7da-abf2a0e91094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:35.554978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:35.703066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b487d-76f4-4b2f-9636-b2930ef95716 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:35.717997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b487d-76f4-4b2f-9636-b2930ef95716 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:35.723309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f01ec5-8ba8-4a81-846d-db0ac27357e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:35.723957 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f01ec5-8ba8-4a81-846d-db0ac27357e6 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:35.724242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f01ec5-8ba8-4a81-846d-db0ac27357e6 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:35.724521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f01ec5-8ba8-4a81-846d-db0ac27357e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:35.733656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f01ec5-8ba8-4a81-846d-db0ac27357e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:35.735278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f01ec5-8ba8-4a81-846d-db0ac27357e6 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:35.744662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 709/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:36.643536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494fd4-2669-4f0d-a54c-2259ff3dbb0c None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:36.644063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494fd4-2669-4f0d-a54c-2259ff3dbb0c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:36.644063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494fd4-2669-4f0d-a54c-2259ff3dbb0c None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:36.645367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:36.650546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618ba31-6c06-4793-9a1a-45c047f40fd5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:36.650958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618ba31-6c06-4793-9a1a-45c047f40fd5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:36.651109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618ba31-6c06-4793-9a1a-45c047f40fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:36.668596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0618ba31-6c06-4793-9a1a-45c047f40fd5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:36.669786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0618ba31-6c06-4793-9a1a-45c047f40fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:36.677803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:36.827234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23494fd4-2669-4f0d-a54c-2259ff3dbb0c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:36.841185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23494fd4-2669-4f0d-a54c-2259ff3dbb0c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:36.846598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf83f3-7ab0-4b71-9ec1-6ed25f2af2c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:36.847226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf83f3-7ab0-4b71-9ec1-6ed25f2af2c7 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:36.847506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf83f3-7ab0-4b71-9ec1-6ed25f2af2c7 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:36.847782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf83f3-7ab0-4b71-9ec1-6ed25f2af2c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:36.856215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf83f3-7ab0-4b71-9ec1-6ed25f2af2c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:36.857774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf83f3-7ab0-4b71-9ec1-6ed25f2af2c7 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:36.867018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 718/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:37.770539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfebd75-ac58-4a7f-9e9f-265797b17f34 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:37.771047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfebd75-ac58-4a7f-9e9f-265797b17f34 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:37.771166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfebd75-ac58-4a7f-9e9f-265797b17f34 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:37.772736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:37.778633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a9678-e001-4e88-9ac9-f6fbf360a72c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:37.779064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a9678-e001-4e88-9ac9-f6fbf360a72c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:37.779172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a9678-e001-4e88-9ac9-f6fbf360a72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:37.796830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c25a9678-e001-4e88-9ac9-f6fbf360a72c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:37.798077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c25a9678-e001-4e88-9ac9-f6fbf360a72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:37.805683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:51:37.955153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfebd75-ac58-4a7f-9e9f-265797b17f34 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:37.969718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfebd75-ac58-4a7f-9e9f-265797b17f34 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:37.975214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112eb6e3-d475-4da3-b0d5-f42a34145a51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:37.975838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eb6e3-d475-4da3-b0d5-f42a34145a51 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:37.976119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eb6e3-d475-4da3-b0d5-f42a34145a51 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:37.976397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112eb6e3-d475-4da3-b0d5-f42a34145a51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:37.984611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112eb6e3-d475-4da3-b0d5-f42a34145a51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:37.986233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112eb6e3-d475-4da3-b0d5-f42a34145a51 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:37.996351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 712/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:39.032993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af34b7-6f8c-41ce-80c9-90da9da8c38e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:39.033573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af34b7-6f8c-41ce-80c9-90da9da8c38e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:39.033573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af34b7-6f8c-41ce-80c9-90da9da8c38e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:39.034864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:39.039642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afcb46-5982-4000-908c-95b2229f250d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:39.040565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afcb46-5982-4000-908c-95b2229f250d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:39.040727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afcb46-5982-4000-908c-95b2229f250d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:39.058714 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f7afcb46-5982-4000-908c-95b2229f250d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:39.059881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f7afcb46-5982-4000-908c-95b2229f250d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:39.067478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:39.215837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af34b7-6f8c-41ce-80c9-90da9da8c38e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:39.230749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af34b7-6f8c-41ce-80c9-90da9da8c38e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:39.236162 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e074ff3-6d5c-49a8-964d-b936c6316114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:39.236794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e074ff3-6d5c-49a8-964d-b936c6316114 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:39.237081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e074ff3-6d5c-49a8-964d-b936c6316114 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:39.237361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e074ff3-6d5c-49a8-964d-b936c6316114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:39.246053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e074ff3-6d5c-49a8-964d-b936c6316114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:39.247634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e074ff3-6d5c-49a8-964d-b936c6316114 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:39.256950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 721/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:40.156177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b9277-c6e0-461e-b654-a2c42bbf8739 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:40.156740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b9277-c6e0-461e-b654-a2c42bbf8739 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:40.156740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b9277-c6e0-461e-b654-a2c42bbf8739 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:40.158117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:40.163006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171af49-9b6c-4534-b1a8-e79fe4a1c38f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:40.163406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171af49-9b6c-4534-b1a8-e79fe4a1c38f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:40.163561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171af49-9b6c-4534-b1a8-e79fe4a1c38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:40.181812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d171af49-9b6c-4534-b1a8-e79fe4a1c38f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:40.182960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d171af49-9b6c-4534-b1a8-e79fe4a1c38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:40.190613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:40.339368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b9277-c6e0-461e-b654-a2c42bbf8739 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:40.353818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b9277-c6e0-461e-b654-a2c42bbf8739 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:40.359107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c3e7f-de91-4b57-992a-e3054958a6f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:40.359730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c3e7f-de91-4b57-992a-e3054958a6f5 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:40.360010 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c3e7f-de91-4b57-992a-e3054958a6f5 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:40.360282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c3e7f-de91-4b57-992a-e3054958a6f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:40.368045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c3e7f-de91-4b57-992a-e3054958a6f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:40.369593 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c3e7f-de91-4b57-992a-e3054958a6f5 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:40.378647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 715/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:41.304757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f361f0-6587-4b11-b087-1c56c140242f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:41.305311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f361f0-6587-4b11-b087-1c56c140242f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:41.305311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f361f0-6587-4b11-b087-1c56c140242f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:41.306688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:41.311707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369c05-86bd-4882-8e9c-ec54708d5f03 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:41.312158 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369c05-86bd-4882-8e9c-ec54708d5f03 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:41.312290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369c05-86bd-4882-8e9c-ec54708d5f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:41.330300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-33369c05-86bd-4882-8e9c-ec54708d5f03 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:41.331466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-33369c05-86bd-4882-8e9c-ec54708d5f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:41.339672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:41.486719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f361f0-6587-4b11-b087-1c56c140242f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:41.501475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f361f0-6587-4b11-b087-1c56c140242f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:41.506945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cf264-216a-4de5-8b58-83a49406dc19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:41.507580 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cf264-216a-4de5-8b58-83a49406dc19 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:41.507870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cf264-216a-4de5-8b58-83a49406dc19 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:41.508154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cf264-216a-4de5-8b58-83a49406dc19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:41.516283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cf264-216a-4de5-8b58-83a49406dc19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:41.517909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cf264-216a-4de5-8b58-83a49406dc19 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:41.528846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 724/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:43.627297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb10138-052b-4eb8-9894-40e4f97378b1 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:43.627789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb10138-052b-4eb8-9894-40e4f97378b1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:43.627894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb10138-052b-4eb8-9894-40e4f97378b1 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:43.629234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:43.635108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7064d3c-8d62-4b6a-9e32-c6b99c489fbc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:43.635512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7064d3c-8d62-4b6a-9e32-c6b99c489fbc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:43.635655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7064d3c-8d62-4b6a-9e32-c6b99c489fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:43.654145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b7064d3c-8d62-4b6a-9e32-c6b99c489fbc None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:43.655339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b7064d3c-8d62-4b6a-9e32-c6b99c489fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:43.663064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:43.700756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb10138-052b-4eb8-9894-40e4f97378b1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:43.702462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bdb10138-052b-4eb8-9894-40e4f97378b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:43.716594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb10138-052b-4eb8-9894-40e4f97378b1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:43.719672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af0134-f025-480b-9e74-967b0ecd59e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:43.720343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af0134-f025-480b-9e74-967b0ecd59e1 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:43.720628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af0134-f025-480b-9e74-967b0ecd59e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:43.720906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af0134-f025-480b-9e74-967b0ecd59e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:43.730176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af0134-f025-480b-9e74-967b0ecd59e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:43.731611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af0134-f025-480b-9e74-967b0ecd59e1 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:43.740891 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 718/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:46.103902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91079fb4-174d-44d1-8795-490455f52bc2 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:46.104474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91079fb4-174d-44d1-8795-490455f52bc2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:46.104474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91079fb4-174d-44d1-8795-490455f52bc2 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:46.105790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:46.111396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a79efd-4a19-4653-aa76-d22c2c7e1db9 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:46.111826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a79efd-4a19-4653-aa76-d22c2c7e1db9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:46.111955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a79efd-4a19-4653-aa76-d22c2c7e1db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:46.129220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-12a79efd-4a19-4653-aa76-d22c2c7e1db9 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:46.130521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-12a79efd-4a19-4653-aa76-d22c2c7e1db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:46.139019 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:46.180826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91079fb4-174d-44d1-8795-490455f52bc2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:46.191248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91079fb4-174d-44d1-8795-490455f52bc2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:46.194737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b78e5-d2cf-4a09-8a88-c8e8d152ed9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:46.195357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b78e5-d2cf-4a09-8a88-c8e8d152ed9a service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:46.195636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b78e5-d2cf-4a09-8a88-c8e8d152ed9a service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:46.196492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b78e5-d2cf-4a09-8a88-c8e8d152ed9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:46.205020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b78e5-d2cf-4a09-8a88-c8e8d152ed9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:46.206460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b78e5-d2cf-4a09-8a88-c8e8d152ed9a service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:46.215445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 727/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:48.887251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79311329-5f47-4841-bea5-b3d0708ef335 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:48.887848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79311329-5f47-4841-bea5-b3d0708ef335 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:48.887848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79311329-5f47-4841-bea5-b3d0708ef335 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:48.889185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:48.894697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe385ae-0bcc-47a8-993e-ce40e8ca8149 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:48.895143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe385ae-0bcc-47a8-993e-ce40e8ca8149 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:48.895254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe385ae-0bcc-47a8-993e-ce40e8ca8149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:48.913589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-bbe385ae-0bcc-47a8-993e-ce40e8ca8149 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:48.914913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bbe385ae-0bcc-47a8-993e-ce40e8ca8149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:48.922510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:51:48.960886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79311329-5f47-4841-bea5-b3d0708ef335 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:48.971961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79311329-5f47-4841-bea5-b3d0708ef335 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:48.975591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706deb4-bdcf-4e15-b858-1036b4b85bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:48.976228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706deb4-bdcf-4e15-b858-1036b4b85bca service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:48.976509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706deb4-bdcf-4e15-b858-1036b4b85bca service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:48.976786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706deb4-bdcf-4e15-b858-1036b4b85bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:48.985663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706deb4-bdcf-4e15-b858-1036b4b85bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:48.987123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706deb4-bdcf-4e15-b858-1036b4b85bca service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:48.996432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 721/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:51.340226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc2ab-124a-4b30-95f2-346882202822 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:51.340811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc2ab-124a-4b30-95f2-346882202822 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:51.340811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dc2ab-124a-4b30-95f2-346882202822 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:51.342181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:51.347392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e70fb8-ce2f-4b1f-b771-93648f92e71c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:51.347818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e70fb8-ce2f-4b1f-b771-93648f92e71c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:51.347923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e70fb8-ce2f-4b1f-b771-93648f92e71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:51.365534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-75e70fb8-ce2f-4b1f-b771-93648f92e71c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:51.366735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-75e70fb8-ce2f-4b1f-b771-93648f92e71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:51.374923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:51.412711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dc2ab-124a-4b30-95f2-346882202822 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:51.423788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dc2ab-124a-4b30-95f2-346882202822 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:51.427240 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d0fc5-5870-4ce5-a8f9-0402a8c0fe3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:51.427863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d0fc5-5870-4ce5-a8f9-0402a8c0fe3e service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:51.428140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d0fc5-5870-4ce5-a8f9-0402a8c0fe3e service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:51.428416 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d0fc5-5870-4ce5-a8f9-0402a8c0fe3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:51.437034 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d0fc5-5870-4ce5-a8f9-0402a8c0fe3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:51.438499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d0fc5-5870-4ce5-a8f9-0402a8c0fe3e service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:51.447590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 730/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:51:53.696082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f47504-da03-4d76-8a27-e6271c84ba06 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:53.696641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f47504-da03-4d76-8a27-e6271c84ba06 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:53.696641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f47504-da03-4d76-8a27-e6271c84ba06 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:53.698004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:53.703307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c9f01-3656-40db-8139-22797cc00a4a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:53.703750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c9f01-3656-40db-8139-22797cc00a4a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:53.703870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c9f01-3656-40db-8139-22797cc00a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:53.722025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9a0c9f01-3656-40db-8139-22797cc00a4a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:53.723220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9a0c9f01-3656-40db-8139-22797cc00a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:53.731594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:53.768720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f47504-da03-4d76-8a27-e6271c84ba06 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:53.780070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f47504-da03-4d76-8a27-e6271c84ba06 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:53.783617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aed38f-bcb8-4f48-9799-b8369b9b9ed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:53.784249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed38f-bcb8-4f48-9799-b8369b9b9ed6 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:53.784513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed38f-bcb8-4f48-9799-b8369b9b9ed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:53.784787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed38f-bcb8-4f48-9799-b8369b9b9ed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:53.794257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aed38f-bcb8-4f48-9799-b8369b9b9ed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:53.797083 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aed38f-bcb8-4f48-9799-b8369b9b9ed6 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:53.804893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 724/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:56.068720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6e4c-c8a6-4184-9a7e-872b5db1bd2b None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:56.069271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6e4c-c8a6-4184-9a7e-872b5db1bd2b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:56.069390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6e4c-c8a6-4184-9a7e-872b5db1bd2b None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:56.071329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:56.076766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fa0e7-1ba1-4f82-af1a-b5adae3a5527 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:56.077174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fa0e7-1ba1-4f82-af1a-b5adae3a5527 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:56.077329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fa0e7-1ba1-4f82-af1a-b5adae3a5527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:56.094776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5c6fa0e7-1ba1-4f82-af1a-b5adae3a5527 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:56.095975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5c6fa0e7-1ba1-4f82-af1a-b5adae3a5527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:56.103708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:56.141507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f6e4c-c8a6-4184-9a7e-872b5db1bd2b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:56.152375 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f6e4c-c8a6-4184-9a7e-872b5db1bd2b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:56.155848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fcd58-9929-4be4-a353-cb75571e8936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:56.156480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fcd58-9929-4be4-a353-cb75571e8936 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:56.156766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fcd58-9929-4be4-a353-cb75571e8936 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:56.157045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fcd58-9929-4be4-a353-cb75571e8936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:56.165809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fcd58-9929-4be4-a353-cb75571e8936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:56.167259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fcd58-9929-4be4-a353-cb75571e8936 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:56.176595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 733/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:51:58.437067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af46d36-edca-40d9-bc94-7b343b2be567 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:58.437639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af46d36-edca-40d9-bc94-7b343b2be567 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:58.437639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af46d36-edca-40d9-bc94-7b343b2be567 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:58.439089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:51:58.444466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8f612-7625-4fc8-803b-57be074ca529 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:58.444872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8f612-7625-4fc8-803b-57be074ca529 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:58.445050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8f612-7625-4fc8-803b-57be074ca529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:58.451761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=70 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:51:58.463682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e7b8f612-7625-4fc8-803b-57be074ca529 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:51:58.464858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e7b8f612-7625-4fc8-803b-57be074ca529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:51:58.472542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:51:58.510566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af46d36-edca-40d9-bc94-7b343b2be567 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:51:58.515896 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=73 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:51:58.522019 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af46d36-edca-40d9-bc94-7b343b2be567 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:51:58.525771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149a7a9-7ea0-4933-b4ce-801e9fe47b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:51:58.531525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149a7a9-7ea0-4933-b4ce-801e9fe47b2a service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:51:58.531525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149a7a9-7ea0-4933-b4ce-801e9fe47b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:51:58.531525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149a7a9-7ea0-4933-b4ce-801e9fe47b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:51:58.535621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149a7a9-7ea0-4933-b4ce-801e9fe47b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:51:58.537035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149a7a9-7ea0-4933-b4ce-801e9fe47b2a service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:51:58.546638 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 727/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:52:00.813816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bc6fd-2a2f-4d35-b61c-dd1b08a62467 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:00.814397 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bc6fd-2a2f-4d35-b61c-dd1b08a62467 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:00.814397 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bc6fd-2a2f-4d35-b61c-dd1b08a62467 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:00.815737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:00.822273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ab77-b2fc-4b53-8936-52b81c721a07 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:00.822698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ab77-b2fc-4b53-8936-52b81c721a07 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:00.822878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ab77-b2fc-4b53-8936-52b81c721a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:00.841387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a630ab77-b2fc-4b53-8936-52b81c721a07 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:00.842597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a630ab77-b2fc-4b53-8936-52b81c721a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:00.850388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:00.887099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bc6fd-2a2f-4d35-b61c-dd1b08a62467 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:00.897974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bc6fd-2a2f-4d35-b61c-dd1b08a62467 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:00.902082 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c4637-9ef4-46e9-a73e-2bcc6a1a1ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:00.902774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c4637-9ef4-46e9-a73e-2bcc6a1a1ffc service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:00.903063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c4637-9ef4-46e9-a73e-2bcc6a1a1ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:00.903346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c4637-9ef4-46e9-a73e-2bcc6a1a1ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:00.912140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c4637-9ef4-46e9-a73e-2bcc6a1a1ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:00.913588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c4637-9ef4-46e9-a73e-2bcc6a1a1ffc service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:00.922880 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 736/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:52:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:52:03.174392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff1574-5503-417b-b02f-f11a8136fa67 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:03.174954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff1574-5503-417b-b02f-f11a8136fa67 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:03.174954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff1574-5503-417b-b02f-f11a8136fa67 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:03.176323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:03.182172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5125ad2-49f7-4b71-b499-d4f47f86bb95 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:03.182573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5125ad2-49f7-4b71-b499-d4f47f86bb95 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:03.182743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5125ad2-49f7-4b71-b499-d4f47f86bb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:03.200375 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a5125ad2-49f7-4b71-b499-d4f47f86bb95 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:03.201568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a5125ad2-49f7-4b71-b499-d4f47f86bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:03.209208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:03.246250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff1574-5503-417b-b02f-f11a8136fa67 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:03.256603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff1574-5503-417b-b02f-f11a8136fa67 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:03.260037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f380ee7-727c-4bc2-912c-3468458a3830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:03.260658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f380ee7-727c-4bc2-912c-3468458a3830 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:03.260935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f380ee7-727c-4bc2-912c-3468458a3830 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:03.261204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f380ee7-727c-4bc2-912c-3468458a3830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:03.269730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f380ee7-727c-4bc2-912c-3468458a3830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:03.271174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f380ee7-727c-4bc2-912c-3468458a3830 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:03.280429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 730/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:52:05.503313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac2938-1384-4769-93f2-3e2582406dac None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:05.503855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac2938-1384-4769-93f2-3e2582406dac None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:05.503855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac2938-1384-4769-93f2-3e2582406dac None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:05.505210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:05.511192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235a7f7-5905-4fbc-a4cd-e78ea1285487 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:05.511623 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235a7f7-5905-4fbc-a4cd-e78ea1285487 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:05.511738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235a7f7-5905-4fbc-a4cd-e78ea1285487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:05.530156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a235a7f7-5905-4fbc-a4cd-e78ea1285487 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:05.531351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a235a7f7-5905-4fbc-a4cd-e78ea1285487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:05.539029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:05.574739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac2938-1384-4769-93f2-3e2582406dac None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:05.585422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac2938-1384-4769-93f2-3e2582406dac None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:05.589024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a1659-1554-4978-8ec6-5c5a87f9cb45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:05.589655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a1659-1554-4978-8ec6-5c5a87f9cb45 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:05.589974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a1659-1554-4978-8ec6-5c5a87f9cb45 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:05.590255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a1659-1554-4978-8ec6-5c5a87f9cb45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:05.599211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a1659-1554-4978-8ec6-5c5a87f9cb45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:05.600629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a1659-1554-4978-8ec6-5c5a87f9cb45 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:05.610531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 739/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:52:15.525056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=17 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:52:15.606314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=17 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:52:37.293558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fbe75-c663-49ee-8994-fcf5aebe1343 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:37.294199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fbe75-c663-49ee-8994-fcf5aebe1343 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:37.294199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fbe75-c663-49ee-8994-fcf5aebe1343 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:37.295513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 732/1471] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:37.300737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18042b-6aa5-4c9b-9286-dd9d14bc7fb7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:37.301204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18042b-6aa5-4c9b-9286-dd9d14bc7fb7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:37.301350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18042b-6aa5-4c9b-9286-dd9d14bc7fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:37.319399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5c18042b-6aa5-4c9b-9286-dd9d14bc7fb7 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:37.320589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5c18042b-6aa5-4c9b-9286-dd9d14bc7fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:37.328262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 740/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:37.374626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea1395-f377-4e92-80d3-48bd0f0e1de2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:37.374914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea1395-f377-4e92-80d3-48bd0f0e1de2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:37.375198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea1395-f377-4e92-80d3-48bd0f0e1de2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:37.376528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 733/1473] 10.222.0.42 () {54 vars in 962 bytes} [Fri Jun 3 09:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:37.381352 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c051-d389-4e0a-8dc4-501d4878d645 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:37.381637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c051-d389-4e0a-8dc4-501d4878d645 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:37.381947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c051-d389-4e0a-8dc4-501d4878d645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:37.400158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a498c051-d389-4e0a-8dc4-501d4878d645 None None] MFA Rules not processed for user `d59a351971e94aa38a22e8113207e5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:37.401338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a498c051-d389-4e0a-8dc4-501d4878d645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:37.407697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 741/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Fri Jun 3 09:52:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:37.415421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea1395-f377-4e92-80d3-48bd0f0e1de2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:37.425745 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea1395-f377-4e92-80d3-48bd0f0e1de2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:37.429177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38001709-d831-46d4-9850-0bfca5247e00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd59a351971e94aa38a22e8113207e5b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:37.429824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38001709-d831-46d4-9850-0bfca5247e00 service glance] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:37.430136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38001709-d831-46d4-9850-0bfca5247e00 service glance] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:37.430410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38001709-d831-46d4-9850-0bfca5247e00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:37.439236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38001709-d831-46d4-9850-0bfca5247e00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:37.440640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38001709-d831-46d4-9850-0bfca5247e00 service glance] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:37.449862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 734/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:52:39.766719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235ab6c-0993-43d1-91e1-9371040aed0d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:39.767278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235ab6c-0993-43d1-91e1-9371040aed0d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:39.767278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235ab6c-0993-43d1-91e1-9371040aed0d None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:39.768695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 742/1476] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:52:39.774715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94138a-ee84-4c11-b7d2-a5bcd153fe92 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:39.775174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94138a-ee84-4c11-b7d2-a5bcd153fe92 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:39.775288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94138a-ee84-4c11-b7d2-a5bcd153fe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:39.792723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4c94138a-ee84-4c11-b7d2-a5bcd153fe92 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:39.795399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4c94138a-ee84-4c11-b7d2-a5bcd153fe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:39.801717 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 735/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:39.825144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235ab6c-0993-43d1-91e1-9371040aed0d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:39.837920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235ab6c-0993-43d1-91e1-9371040aed0d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:39.842263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51409cc-177f-4e11-9d79-cfa028f000c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:39.842901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51409cc-177f-4e11-9d79-cfa028f000c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:39.843184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51409cc-177f-4e11-9d79-cfa028f000c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:39.843459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51409cc-177f-4e11-9d79-cfa028f000c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:39.844182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51409cc-177f-4e11-9d79-cfa028f000c7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:39.845860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-c51409cc-177f-4e11-9d79-cfa028f000c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:52:39.845860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jun 03 09:52:39.845860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: Jun 03 09:52:39.847010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51409cc-177f-4e11-9d79-cfa028f000c7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:39.865753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 743/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Fri Jun 3 09:52:39 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:52:42.119345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ecb05-fc10-494e-a829-97530af1cd95 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:42.119906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ecb05-fc10-494e-a829-97530af1cd95 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:42.119906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ecb05-fc10-494e-a829-97530af1cd95 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:42.121289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 736/1479] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:42.126707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f23645-ee60-4303-852f-ee0d267e4312 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:42.127108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f23645-ee60-4303-852f-ee0d267e4312 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:42.127260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f23645-ee60-4303-852f-ee0d267e4312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:42.144998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d8f23645-ee60-4303-852f-ee0d267e4312 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:42.146487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d8f23645-ee60-4303-852f-ee0d267e4312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:42.154199 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 744/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:42.197716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ecb05-fc10-494e-a829-97530af1cd95 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:42.211809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ecb05-fc10-494e-a829-97530af1cd95 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:42.217162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284e8c6-7aa2-420b-a9bf-297737105355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:42.217715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284e8c6-7aa2-420b-a9bf-297737105355 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:42.218035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284e8c6-7aa2-420b-a9bf-297737105355 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:42.218316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284e8c6-7aa2-420b-a9bf-297737105355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:42.226869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284e8c6-7aa2-420b-a9bf-297737105355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:42.228437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284e8c6-7aa2-420b-a9bf-297737105355 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:42.237721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 737/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:52:44.628290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfa3c7-0600-4a8e-ad03-e894570acab6 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:44.628861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfa3c7-0600-4a8e-ad03-e894570acab6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:44.628861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfa3c7-0600-4a8e-ad03-e894570acab6 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:44.630232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 745/1482] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:44.635939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bee5b5-39db-48da-8953-01055168e831 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:44.636337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bee5b5-39db-48da-8953-01055168e831 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:44.636504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bee5b5-39db-48da-8953-01055168e831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:44.654657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b8bee5b5-39db-48da-8953-01055168e831 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:44.655855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b8bee5b5-39db-48da-8953-01055168e831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:44.663867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 738/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:44.699413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfa3c7-0600-4a8e-ad03-e894570acab6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:44.710560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfa3c7-0600-4a8e-ad03-e894570acab6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:44.714284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73875e4c-d0cc-4616-a472-9d2794fa5367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:44.714931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875e4c-d0cc-4616-a472-9d2794fa5367 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:44.715214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875e4c-d0cc-4616-a472-9d2794fa5367 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:44.715490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73875e4c-d0cc-4616-a472-9d2794fa5367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:44.726902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73875e4c-d0cc-4616-a472-9d2794fa5367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:44.728326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73875e4c-d0cc-4616-a472-9d2794fa5367 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:44.737668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 746/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:52:47.027772 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44268e22-c823-4045-b7ce-2e4480072dee None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:47.028328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44268e22-c823-4045-b7ce-2e4480072dee None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:47.028328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44268e22-c823-4045-b7ce-2e4480072dee None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:47.029621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 739/1485] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:47.034797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2081995-8a43-4e16-8b1a-b5b934d26b57 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:47.035201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2081995-8a43-4e16-8b1a-b5b934d26b57 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:47.035379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2081995-8a43-4e16-8b1a-b5b934d26b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:47.053844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b2081995-8a43-4e16-8b1a-b5b934d26b57 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:47.055078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b2081995-8a43-4e16-8b1a-b5b934d26b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:47.062831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 747/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:47.102128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44268e22-c823-4045-b7ce-2e4480072dee None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:47.112755 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44268e22-c823-4045-b7ce-2e4480072dee None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:47.116679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c40b5-f022-4227-b8e0-a9d7e08032b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:47.117316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c40b5-f022-4227-b8e0-a9d7e08032b4 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:47.117617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c40b5-f022-4227-b8e0-a9d7e08032b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:47.117895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c40b5-f022-4227-b8e0-a9d7e08032b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:47.126829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c40b5-f022-4227-b8e0-a9d7e08032b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:47.128275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c40b5-f022-4227-b8e0-a9d7e08032b4 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:47.137776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 740/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:52:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:52:49.507461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57f8e9-8c88-45c9-ae5a-dfca85267d5f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:49.508003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57f8e9-8c88-45c9-ae5a-dfca85267d5f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:49.508003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57f8e9-8c88-45c9-ae5a-dfca85267d5f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:49.509381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 748/1488] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:49.514819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11332bec-9500-4230-9012-55796cc6289e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:49.515232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11332bec-9500-4230-9012-55796cc6289e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:49.515384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11332bec-9500-4230-9012-55796cc6289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:49.534051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-11332bec-9500-4230-9012-55796cc6289e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:49.535332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-11332bec-9500-4230-9012-55796cc6289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:49.544597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 741/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:49.585771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57f8e9-8c88-45c9-ae5a-dfca85267d5f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:49.597829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57f8e9-8c88-45c9-ae5a-dfca85267d5f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:49.602833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700376d-9ef2-4294-b75b-1f60e7506724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:49.603504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700376d-9ef2-4294-b75b-1f60e7506724 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:49.603807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700376d-9ef2-4294-b75b-1f60e7506724 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:49.604085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700376d-9ef2-4294-b75b-1f60e7506724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:49.614360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700376d-9ef2-4294-b75b-1f60e7506724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:49.616526 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700376d-9ef2-4294-b75b-1f60e7506724 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:49.627723 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 749/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:52:52.354336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8098995-5049-4d3f-a193-8b94317a33f1 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:52.354893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8098995-5049-4d3f-a193-8b94317a33f1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:52.354893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8098995-5049-4d3f-a193-8b94317a33f1 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:52.356324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 742/1491] 10.222.0.42 () {54 vars in 926 bytes} [Fri Jun 3 09:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:52:52.361414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383050b-04fa-46d0-ad9d-399e94ab4f21 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:52.361833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383050b-04fa-46d0-ad9d-399e94ab4f21 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:52.362010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383050b-04fa-46d0-ad9d-399e94ab4f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:52.379775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a383050b-04fa-46d0-ad9d-399e94ab4f21 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:52:52.381558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a383050b-04fa-46d0-ad9d-399e94ab4f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:52:52.389373 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 750/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Fri Jun 3 09:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:52:52.427789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8098995-5049-4d3f-a193-8b94317a33f1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:52:52.438266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8098995-5049-4d3f-a193-8b94317a33f1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:52:52.441725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39933f44-171c-4fa2-94ee-bcb781446275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:52:52.442388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39933f44-171c-4fa2-94ee-bcb781446275 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:52:52.442678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39933f44-171c-4fa2-94ee-bcb781446275 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:52:52.443577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39933f44-171c-4fa2-94ee-bcb781446275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:52:52.452267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39933f44-171c-4fa2-94ee-bcb781446275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:52:52.453684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39933f44-171c-4fa2-94ee-bcb781446275 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:52:52.462980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 743/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:52:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:53:02.374809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=33 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:53:02.458866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=33 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:54:25.943581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa673a96-07d1-4629-b98f-bd1600b0dd6d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:25.944175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa673a96-07d1-4629-b98f-bd1600b0dd6d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:25.944175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa673a96-07d1-4629-b98f-bd1600b0dd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:25.961442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-aa673a96-07d1-4629-b98f-bd1600b0dd6d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:54:25.962645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aa673a96-07d1-4629-b98f-bd1600b0dd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:54:25.970707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 751/1494] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:54:25.979515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f894c9f-2259-4e3e-b7d0-3f790ee3b618 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:25.980000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f894c9f-2259-4e3e-b7d0-3f790ee3b618 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:25.980104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f894c9f-2259-4e3e-b7d0-3f790ee3b618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:25.997982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1f894c9f-2259-4e3e-b7d0-3f790ee3b618 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:54:25.999194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1f894c9f-2259-4e3e-b7d0-3f790ee3b618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:54:26.006852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 744/1495] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:54:26.011227 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa673a96-07d1-4629-b98f-bd1600b0dd6d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:26.020980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa673a96-07d1-4629-b98f-bd1600b0dd6d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:26.025370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:26.026045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.026332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.026609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.027338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:54:26.027647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:26.028649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:54:26.028649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jun 03 09:54:26.028649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: Jun 03 09:54:26.029521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:26.041736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 752/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:54:26.050233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f894c9f-2259-4e3e-b7d0-3f790ee3b618 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:26.059192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f894c9f-2259-4e3e-b7d0-3f790ee3b618 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:26.063760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8f19a-3f42-4688-bb56-5978cf075a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:26.064403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f19a-3f42-4688-bb56-5978cf075a05 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.064686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f19a-3f42-4688-bb56-5978cf075a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.064965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8f19a-3f42-4688-bb56-5978cf075a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.065641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8f19a-3f42-4688-bb56-5978cf075a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:26.067518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-98c8f19a-3f42-4688-bb56-5978cf075a05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:54:26.067518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jun 03 09:54:26.067518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: Jun 03 09:54:26.068713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8f19a-3f42-4688-bb56-5978cf075a05 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:26.153353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 745/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:54:26 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:54:26.157490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:26.167872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91af28d-8435-438b-a0a4-90fc8af41cd9 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:26.222333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6d054-381a-47c9-908d-9e222ab5fa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:26.222925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6d054-381a-47c9-908d-9e222ab5fa7d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.223208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6d054-381a-47c9-908d-9e222ab5fa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.223490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a6d054-381a-47c9-908d-9e222ab5fa7d admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.224182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6d054-381a-47c9-908d-9e222ab5fa7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:26.225181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-10a6d054-381a-47c9-908d-9e222ab5fa7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:54:26.225181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jun 03 09:54:26.225181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: Jun 03 09:54:26.226424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a6d054-381a-47c9-908d-9e222ab5fa7d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:26.321603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 753/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:54:26 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:54:26.325629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8f19a-3f42-4688-bb56-5978cf075a05 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:26.336552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8f19a-3f42-4688-bb56-5978cf075a05 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:26.341204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c64a8-7cd8-4669-8c23-e698c4c15a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:26.341859 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c64a8-7cd8-4669-8c23-e698c4c15a6d admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.342177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c64a8-7cd8-4669-8c23-e698c4c15a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.342455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c64a8-7cd8-4669-8c23-e698c4c15a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.343170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c64a8-7cd8-4669-8c23-e698c4c15a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:26.344177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-5d4c64a8-7cd8-4669-8c23-e698c4c15a6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:54:26.344177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jun 03 09:54:26.344177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: Jun 03 09:54:26.345064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c64a8-7cd8-4669-8c23-e698c4c15a6d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:26.358470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 746/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:54:26.362312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6d054-381a-47c9-908d-9e222ab5fa7d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:26.372139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a6d054-381a-47c9-908d-9e222ab5fa7d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:26.376710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec334f75-23f7-40d7-a75d-79cb2a1a80eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:26.377399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec334f75-23f7-40d7-a75d-79cb2a1a80eb admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.377687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec334f75-23f7-40d7-a75d-79cb2a1a80eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.378006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec334f75-23f7-40d7-a75d-79cb2a1a80eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.378736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec334f75-23f7-40d7-a75d-79cb2a1a80eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:26.379737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-ec334f75-23f7-40d7-a75d-79cb2a1a80eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:54:26.379737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jun 03 09:54:26.379737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: Jun 03 09:54:26.380616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec334f75-23f7-40d7-a75d-79cb2a1a80eb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:26.394460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 754/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:54:26.398332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c64a8-7cd8-4669-8c23-e698c4c15a6d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:26.408425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c64a8-7cd8-4669-8c23-e698c4c15a6d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:26.412808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45600782-cdd0-4614-a1dc-04617b838643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:26.413667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45600782-cdd0-4614-a1dc-04617b838643 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.413977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45600782-cdd0-4614-a1dc-04617b838643 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.414251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45600782-cdd0-4614-a1dc-04617b838643 admin admin] PATH_INFO: `/v3/projects/10b71a4c0d1640199b9aca7bd75b53c1/users/8258618a020e4eb58f293dbdfedd6933/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.428435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45600782-cdd0-4614-a1dc-04617b838643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b71a4c0d1640199b9aca7bd75b53c1, user_id=8258618a020e4eb58f293dbdfedd6933, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:26.429454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-45600782-cdd0-4614-a1dc-04617b838643 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:54:26.429454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jun 03 09:54:26.429454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: Jun 03 09:54:26.431395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45600782-cdd0-4614-a1dc-04617b838643 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:26.575004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 747/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:54:26 2022] PUT /identity/v3/projects/10b71a4c0d1640199b9aca7bd75b53c1/users/8258618a020e4eb58f293dbdfedd6933/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:54:26.587501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675c640-8e57-4cbf-8907-aceefc131115 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.587801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675c640-8e57-4cbf-8907-aceefc131115 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.588085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675c640-8e57-4cbf-8907-aceefc131115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.614780 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5675c640-8e57-4cbf-8907-aceefc131115 None None] MFA Rules not processed for user `8258618a020e4eb58f293dbdfedd6933`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:54:26.616093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5675c640-8e57-4cbf-8907-aceefc131115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:54:26.677902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 755/1502] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:54:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:54:26.691503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad0381-4ddf-4008-94c1-52dc30057948 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.691799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad0381-4ddf-4008-94c1-52dc30057948 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.692079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad0381-4ddf-4008-94c1-52dc30057948 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.693462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 748/1503] 10.222.0.42 () {54 vars in 963 bytes} [Fri Jun 3 09:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:54:26.698506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9d18-0749-40e3-a860-3746cf6011d9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.698802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9d18-0749-40e3-a860-3746cf6011d9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.699109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a9d18-0749-40e3-a860-3746cf6011d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.719068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-945a9d18-0749-40e3-a860-3746cf6011d9 None None] MFA Rules not processed for user `aee48298b3834984a12fe2fe0a6f9a77`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:54:26.720248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-945a9d18-0749-40e3-a860-3746cf6011d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:54:26.763816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:54:26.772159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 756/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Fri Jun 3 09:54:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:54:26.779885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad0381-4ddf-4008-94c1-52dc30057948 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:26.790253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad0381-4ddf-4008-94c1-52dc30057948 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:26.793757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033b055-cceb-4ce6-b8a5-cfedb8b68d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:26.794416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033b055-cceb-4ce6-b8a5-cfedb8b68d4b service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.794704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033b055-cceb-4ce6-b8a5-cfedb8b68d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.794985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033b055-cceb-4ce6-b8a5-cfedb8b68d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.803307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033b055-cceb-4ce6-b8a5-cfedb8b68d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:26.804732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033b055-cceb-4ce6-b8a5-cfedb8b68d4b service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:26.815143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 749/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:54:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:54:26.859724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512baba-df85-4111-bbb7-8ed317b7d951 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.870198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512baba-df85-4111-bbb7-8ed317b7d951 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.870198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512baba-df85-4111-bbb7-8ed317b7d951 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.870198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 757/1506] 10.222.0.42 () {52 vars in 829 bytes} [Fri Jun 3 09:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:54:26.880624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033b055-cceb-4ce6-b8a5-cfedb8b68d4b service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:26.942893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033b055-cceb-4ce6-b8a5-cfedb8b68d4b service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:26.947689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:26.948557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:26.948845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:26.949139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] PATH_INFO: `/v3/users/8258618a020e4eb58f293dbdfedd6933` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:26.950390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8258618a020e4eb58f293dbdfedd6933)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:26.951514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:54:26.951514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jun 03 09:54:26.951514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: Jun 03 09:54:26.952722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:27.198132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8258618a020e4eb58f293dbdfedd6933'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:27.198765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8258618a020e4eb58f293dbdfedd6933'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:27.216802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8258618a020e4eb58f293dbdfedd6933'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:27.230825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8258618a020e4eb58f293dbdfedd6933'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:27.234721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:54:27.317488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8258618a020e4eb58f293dbdfedd6933'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:27.323579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 750/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:54:26 2022] DELETE /identity/v3/users/8258618a020e4eb58f293dbdfedd6933 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:54:27.328184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7be2a7-f1df-41a4-94af-35f5971ed865 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:27.328650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7be2a7-f1df-41a4-94af-35f5971ed865 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:27.328745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7be2a7-f1df-41a4-94af-35f5971ed865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:27.346776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-dc7be2a7-f1df-41a4-94af-35f5971ed865 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:54:27.347958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dc7be2a7-f1df-41a4-94af-35f5971ed865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:54:27.404284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 758/1508] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:54:27.414781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:27.416181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:54:27.471710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7dbdf-485f-4e0e-a391-6381e2f15495 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:27.475140 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d4f28-a039-4b8d-8d64-63ae5129eb51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:27.475778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d4f28-a039-4b8d-8d64-63ae5129eb51 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:27.476063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d4f28-a039-4b8d-8d64-63ae5129eb51 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:27.476347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d4f28-a039-4b8d-8d64-63ae5129eb51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:27.485098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d4f28-a039-4b8d-8d64-63ae5129eb51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:27.487189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d4f28-a039-4b8d-8d64-63ae5129eb51 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:27.496358 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 751/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:54:27.988358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7be2a7-f1df-41a4-94af-35f5971ed865 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:54:27.989839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dc7be2a7-f1df-41a4-94af-35f5971ed865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:54:28.003626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7be2a7-f1df-41a4-94af-35f5971ed865 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:54:28.008951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:54:28.009862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:54:28.010298 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:54:28.010528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] PATH_INFO: `/v3/projects/10b71a4c0d1640199b9aca7bd75b53c1` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:54:28.012621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b71a4c0d1640199b9aca7bd75b53c1)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:54:28.013698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:54:28.013698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jun 03 09:54:28.013698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: Jun 03 09:54:28.015102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:54:28.186689 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b71a4c0d1640199b9aca7bd75b53c1', 'request_id': 'req-199cc666-2c17-4063-b84e-9465204a0f86'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:28.193397 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b71a4c0d1640199b9aca7bd75b53c1', 'request_id': 'req-199cc666-2c17-4063-b84e-9465204a0f86'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:28.270544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b71a4c0d1640199b9aca7bd75b53c1', 'request_id': 'req-199cc666-2c17-4063-b84e-9465204a0f86'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:28.288913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] The token cache is being invalidate because project 10b71a4c0d1640199b9aca7bd75b53c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:54:28.289200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:28.289833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-199cc666-2c17-4063-b84e-9465204a0f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:54:28.291494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 759/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:54:27 2022] DELETE /identity/v3/projects/10b71a4c0d1640199b9aca7bd75b53c1 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:54:37.492285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:54:38.285945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:56:09.018733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb744e-3e65-4ba6-9d7c-c1f890824d8b None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:09.019324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb744e-3e65-4ba6-9d7c-c1f890824d8b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:09.019324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb744e-3e65-4ba6-9d7c-c1f890824d8b None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:09.020730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:56:09.025550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d621-6df7-4710-94f4-e3b28a100f38 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:09.026011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d621-6df7-4710-94f4-e3b28a100f38 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:09.026126 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d621-6df7-4710-94f4-e3b28a100f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:09.043725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-dab8d621-6df7-4710-94f4-e3b28a100f38 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:09.044898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dab8d621-6df7-4710-94f4-e3b28a100f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:09.105572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:09.271317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb744e-3e65-4ba6-9d7c-c1f890824d8b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:56:09.272952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3efb744e-3e65-4ba6-9d7c-c1f890824d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:09.346422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb744e-3e65-4ba6-9d7c-c1f890824d8b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:56:09.351639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc54d41d-6bda-43bb-ade8-ce7e56512a49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:56:09.352274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54d41d-6bda-43bb-ade8-ce7e56512a49 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:09.352557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54d41d-6bda-43bb-ade8-ce7e56512a49 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:09.352833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc54d41d-6bda-43bb-ade8-ce7e56512a49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:09.360712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54d41d-6bda-43bb-ade8-ce7e56512a49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:56:09.362319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc54d41d-6bda-43bb-ade8-ce7e56512a49 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:56:09.371343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 753/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:56:09.771045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271975d-90d9-433a-8be8-ad16b1e46a5d None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:09.771338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271975d-90d9-433a-8be8-ad16b1e46a5d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:09.771643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271975d-90d9-433a-8be8-ad16b1e46a5d None None] PATH_INFO: `/v3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:09.773130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 761/1514] 10.222.0.59 () {54 vars in 925 bytes} [Fri Jun 3 09:56:09 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:56:09.779339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80b194-066a-4094-b67d-b6a4fa3a1cf4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:09.779634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80b194-066a-4094-b67d-b6a4fa3a1cf4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:09.779918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80b194-066a-4094-b67d-b6a4fa3a1cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:09.799707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0b80b194-066a-4094-b67d-b6a4fa3a1cf4 None None] MFA Rules not processed for user `566c0c4bdcaf4c13b060af4bbff5bc04`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:09.801003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0b80b194-066a-4094-b67d-b6a4fa3a1cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:09.863737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 754/1515] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Jun 3 09:56:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:10.246362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687973bc-e5af-4dac-9235-98f425610462 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:10.246910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687973bc-e5af-4dac-9235-98f425610462 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:10.246910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687973bc-e5af-4dac-9235-98f425610462 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:10.248205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 762/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:56:10.252896 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9c317-b703-4f38-92d0-5b9c6daf180f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:10.253335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9c317-b703-4f38-92d0-5b9c6daf180f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:10.253491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9c317-b703-4f38-92d0-5b9c6daf180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:10.271426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-50c9c317-b703-4f38-92d0-5b9c6daf180f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:10.272615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-50c9c317-b703-4f38-92d0-5b9c6daf180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:10.280925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 755/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:10.428790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687973bc-e5af-4dac-9235-98f425610462 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:56:10.442799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687973bc-e5af-4dac-9235-98f425610462 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:56:10.447965 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608439a-b9f0-470d-b381-05a2ff8cc1fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:56:10.448587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608439a-b9f0-470d-b381-05a2ff8cc1fe service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:10.448866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608439a-b9f0-470d-b381-05a2ff8cc1fe service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:10.449138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608439a-b9f0-470d-b381-05a2ff8cc1fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:10.457115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608439a-b9f0-470d-b381-05a2ff8cc1fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:56:10.458716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608439a-b9f0-470d-b381-05a2ff8cc1fe service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:56:10.468348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 763/1518] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:56:13.919894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab220296-5c26-47b4-b12d-396bb4d80e9c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:13.920503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab220296-5c26-47b4-b12d-396bb4d80e9c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:13.921117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab220296-5c26-47b4-b12d-396bb4d80e9c None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:13.922464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 756/1519] 10.222.0.42 () {54 vars in 960 bytes} [Fri Jun 3 09:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:56:13.928279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385647d-b14c-4bfc-890e-a3c40d787611 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:13.928690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385647d-b14c-4bfc-890e-a3c40d787611 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:13.928854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385647d-b14c-4bfc-890e-a3c40d787611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:13.948447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0385647d-b14c-4bfc-890e-a3c40d787611 None None] MFA Rules not processed for user `566c0c4bdcaf4c13b060af4bbff5bc04`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:13.949633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0385647d-b14c-4bfc-890e-a3c40d787611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:13.958140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 764/1520] 10.222.0.42 () {58 vars in 1057 bytes} [Fri Jun 3 09:56:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:13.965975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab220296-5c26-47b4-b12d-396bb4d80e9c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:56:13.979886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab220296-5c26-47b4-b12d-396bb4d80e9c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:56:13.985839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195e732-fa0a-4163-88d3-396a1fa3817f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:56:13.986496 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195e732-fa0a-4163-88d3-396a1fa3817f service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:13.986788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195e732-fa0a-4163-88d3-396a1fa3817f service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:13.987059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195e732-fa0a-4163-88d3-396a1fa3817f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:13.995335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195e732-fa0a-4163-88d3-396a1fa3817f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:56:13.997004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195e732-fa0a-4163-88d3-396a1fa3817f service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:56:14.006509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 757/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:56:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:56:21.729663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f29be6-9619-499b-b8db-4a7f08832959 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:21.730170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f29be6-9619-499b-b8db-4a7f08832959 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:21.730276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f29be6-9619-499b-b8db-4a7f08832959 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:21.731602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:56:21.736912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28124a7-918a-4d50-9511-3fadd59a6a65 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:21.737316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28124a7-918a-4d50-9511-3fadd59a6a65 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:21.737481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28124a7-918a-4d50-9511-3fadd59a6a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:21.755374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a28124a7-918a-4d50-9511-3fadd59a6a65 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:21.756600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a28124a7-918a-4d50-9511-3fadd59a6a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:21.765134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:21.919862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f29be6-9619-499b-b8db-4a7f08832959 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:56:21.936215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f29be6-9619-499b-b8db-4a7f08832959 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:56:21.944061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a60aeb-0262-4282-ba2d-06e21e35add9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:56:21.944061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60aeb-0262-4282-ba2d-06e21e35add9 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:21.944061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60aeb-0262-4282-ba2d-06e21e35add9 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:21.944061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60aeb-0262-4282-ba2d-06e21e35add9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:21.950781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a60aeb-0262-4282-ba2d-06e21e35add9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:56:21.952923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a60aeb-0262-4282-ba2d-06e21e35add9 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:56:21.962104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 766/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:56:22.831637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75046c-38b8-498c-bf60-fce83bee2af3 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:22.832173 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75046c-38b8-498c-bf60-fce83bee2af3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:22.832284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75046c-38b8-498c-bf60-fce83bee2af3 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:22.833575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 09:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:56:22.838296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465e00-96d2-491b-8473-e962abbfff17 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:22.838687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465e00-96d2-491b-8473-e962abbfff17 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:22.838845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465e00-96d2-491b-8473-e962abbfff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:22.856788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2b465e00-96d2-491b-8473-e962abbfff17 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:22.858012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2b465e00-96d2-491b-8473-e962abbfff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:22.865534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 09:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:23.015263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75046c-38b8-498c-bf60-fce83bee2af3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:56:23.029558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75046c-38b8-498c-bf60-fce83bee2af3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:56:23.034918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b425c-985d-401e-a75b-fd86cbdfba0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:56:23.035547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b425c-985d-401e-a75b-fd86cbdfba0e service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:23.035832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b425c-985d-401e-a75b-fd86cbdfba0e service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:23.036112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b425c-985d-401e-a75b-fd86cbdfba0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:23.044029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b425c-985d-401e-a75b-fd86cbdfba0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:56:23.045645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b425c-985d-401e-a75b-fd86cbdfba0e service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:56:23.054875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 760/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:56:27.209348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b28b77-c439-4903-b1ba-69be9d7153e7 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:27.209978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b28b77-c439-4903-b1ba-69be9d7153e7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:27.209978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b28b77-c439-4903-b1ba-69be9d7153e7 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:27.211297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:56:27.216472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861200f2-002c-46e7-80cc-325787d4bb31 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:27.216900 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861200f2-002c-46e7-80cc-325787d4bb31 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:27.217070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861200f2-002c-46e7-80cc-325787d4bb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:27.235433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-861200f2-002c-46e7-80cc-325787d4bb31 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:27.236603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-861200f2-002c-46e7-80cc-325787d4bb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:27.244251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:27.280462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b28b77-c439-4903-b1ba-69be9d7153e7 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:56:27.282165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-01b28b77-c439-4903-b1ba-69be9d7153e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:27.337793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b28b77-c439-4903-b1ba-69be9d7153e7 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:56:27.341249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4871016-8eeb-4c0c-9a12-30af3f58f71f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:56:27.341971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4871016-8eeb-4c0c-9a12-30af3f58f71f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:27.342257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4871016-8eeb-4c0c-9a12-30af3f58f71f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:27.342537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4871016-8eeb-4c0c-9a12-30af3f58f71f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:27.351249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4871016-8eeb-4c0c-9a12-30af3f58f71f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:56:27.352651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4871016-8eeb-4c0c-9a12-30af3f58f71f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:56:27.362308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 769/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:56:29.584461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba620-baf7-49bc-9493-c31309b4c0b5 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:29.585027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba620-baf7-49bc-9493-c31309b4c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:29.585027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ba620-baf7-49bc-9493-c31309b4c0b5 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:29.586466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:56:29.592665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c62465-dabb-4683-a0a0-d301d704a73b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:29.593114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c62465-dabb-4683-a0a0-d301d704a73b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:29.593219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c62465-dabb-4683-a0a0-d301d704a73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:29.612673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-54c62465-dabb-4683-a0a0-d301d704a73b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:29.613867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-54c62465-dabb-4683-a0a0-d301d704a73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:29.621442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:29.657143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba620-baf7-49bc-9493-c31309b4c0b5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:56:29.667595 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ba620-baf7-49bc-9493-c31309b4c0b5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:56:29.671065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065d779-64ea-44a5-a87d-f7ca05c62508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:56:29.671692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065d779-64ea-44a5-a87d-f7ca05c62508 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:29.671977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065d779-64ea-44a5-a87d-f7ca05c62508 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:29.672259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065d779-64ea-44a5-a87d-f7ca05c62508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:29.681480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065d779-64ea-44a5-a87d-f7ca05c62508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:56:29.683071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065d779-64ea-44a5-a87d-f7ca05c62508 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:56:29.692813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 763/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:56:33.377031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ca409-18bb-41ee-aa40-86cca1ca5c1e None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:33.377573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ca409-18bb-41ee-aa40-86cca1ca5c1e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:33.377573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ca409-18bb-41ee-aa40-86cca1ca5c1e None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:33.378977 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:56:33.384131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ec52-8d7f-4ce3-96d7-d703063cb21f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:33.384574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ec52-8d7f-4ce3-96d7-d703063cb21f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:33.384680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0ec52-8d7f-4ce3-96d7-d703063cb21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:33.402663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2af0ec52-8d7f-4ce3-96d7-d703063cb21f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:33.403854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2af0ec52-8d7f-4ce3-96d7-d703063cb21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:33.412007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:33.677225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ca409-18bb-41ee-aa40-86cca1ca5c1e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:56:33.678907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-924ca409-18bb-41ee-aa40-86cca1ca5c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:33.735827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ca409-18bb-41ee-aa40-86cca1ca5c1e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:56:33.739337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd3fb4-78c2-49c6-b454-b25cd3f9df2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:56:33.739979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd3fb4-78c2-49c6-b454-b25cd3f9df2c service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:33.740263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd3fb4-78c2-49c6-b454-b25cd3f9df2c service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:33.740539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd3fb4-78c2-49c6-b454-b25cd3f9df2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:33.749619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd3fb4-78c2-49c6-b454-b25cd3f9df2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:56:33.751111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd3fb4-78c2-49c6-b454-b25cd3f9df2c service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:56:33.755950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:56:33.761469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 772/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:56:36.023981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4267-8435-47f0-8436-ba8572f58439 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:36.024505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4267-8435-47f0-8436-ba8572f58439 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:36.024505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4267-8435-47f0-8436-ba8572f58439 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:36.025866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 09:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:56:36.031613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6064a96-f3f4-4305-a859-daca4778390e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:36.032034 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6064a96-f3f4-4305-a859-daca4778390e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:36.032176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6064a96-f3f4-4305-a859-daca4778390e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:36.049781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b6064a96-f3f4-4305-a859-daca4778390e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:56:36.050990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b6064a96-f3f4-4305-a859-daca4778390e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:36.058709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 09:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:56:36.326603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d4267-8435-47f0-8436-ba8572f58439 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:56:36.328346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-105d4267-8435-47f0-8436-ba8572f58439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:56:36.341256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d4267-8435-47f0-8436-ba8572f58439 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:56:36.345379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abb6ed-64ee-46b7-9bad-c75a55319825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:56:36.346044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abb6ed-64ee-46b7-9bad-c75a55319825 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:56:36.346331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abb6ed-64ee-46b7-9bad-c75a55319825 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:56:36.346608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abb6ed-64ee-46b7-9bad-c75a55319825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:56:36.355686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57abb6ed-64ee-46b7-9bad-c75a55319825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:56:36.356794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57abb6ed-64ee-46b7-9bad-c75a55319825 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:56:36.366273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 766/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:56:46.044622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:56:46.362094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:07.548231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879e4d7-4980-4382-8e6f-c96a001068ad None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.548886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879e4d7-4980-4382-8e6f-c96a001068ad None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.548886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879e4d7-4980-4382-8e6f-c96a001068ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.577250 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b879e4d7-4980-4382-8e6f-c96a001068ad None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:07.578907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b879e4d7-4980-4382-8e6f-c96a001068ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:07.588919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 774/1540] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:07.598460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da963-e4c2-420c-8213-87aa59934bd6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.599057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da963-e4c2-420c-8213-87aa59934bd6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.599057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da963-e4c2-420c-8213-87aa59934bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.624971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d48da963-e4c2-420c-8213-87aa59934bd6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:07.626215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d48da963-e4c2-420c-8213-87aa59934bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:07.633745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3da8e-3ddc-49fa-b3b7-66e85377423a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.634439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3da8e-3ddc-49fa-b3b7-66e85377423a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.635010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3da8e-3ddc-49fa-b3b7-66e85377423a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.644276 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 767/1541] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:07.660691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48da963-e4c2-420c-8213-87aa59934bd6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:07.669838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1fc3da8e-3ddc-49fa-b3b7-66e85377423a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:07.671146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1fc3da8e-3ddc-49fa-b3b7-66e85377423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:07.675449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48da963-e4c2-420c-8213-87aa59934bd6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:07.686811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:07.688739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.688739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.688739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.689013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:07.689096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:07.689177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 775/1542] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:58:07.690141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:58:07.690141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jun 03 09:58:07.690141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: Jun 03 09:58:07.691049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:07.697821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad49a2-0b3f-4276-82ab-7d69de209686 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.698266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad49a2-0b3f-4276-82ab-7d69de209686 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.698462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad49a2-0b3f-4276-82ab-7d69de209686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.723273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 768/1543] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:58:07.744807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-28ad49a2-0b3f-4276-82ab-7d69de209686 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:07.745030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:07.745127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-28ad49a2-0b3f-4276-82ab-7d69de209686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:07.759288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6146e97-0811-4cff-8073-9d0218d3d1a1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:07.761006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 776/1544] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:07.763400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b339d-6594-4270-8352-b0aafcab1b22 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.763717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b339d-6594-4270-8352-b0aafcab1b22 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.764319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b339d-6594-4270-8352-b0aafcab1b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.773657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40379b-f8aa-446a-b351-83ba04c730e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:07.778870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40379b-f8aa-446a-b351-83ba04c730e1 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.778870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40379b-f8aa-446a-b351-83ba04c730e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.778870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40379b-f8aa-446a-b351-83ba04c730e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.778870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40379b-f8aa-446a-b351-83ba04c730e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:07.778870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40379b-f8aa-446a-b351-83ba04c730e1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:07.799031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-690b339d-6594-4270-8352-b0aafcab1b22 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:07.800339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-690b339d-6594-4270-8352-b0aafcab1b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:07.821402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 777/1545] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:07.822458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b339d-6594-4270-8352-b0aafcab1b22 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:07.837432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b339d-6594-4270-8352-b0aafcab1b22 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:07.850268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebf1f9-d11b-4197-ba5d-9a93120bd293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:07.850268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebf1f9-d11b-4197-ba5d-9a93120bd293 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.850268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebf1f9-d11b-4197-ba5d-9a93120bd293 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.850268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebf1f9-d11b-4197-ba5d-9a93120bd293 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.851159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebf1f9-d11b-4197-ba5d-9a93120bd293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:07.851478 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebf1f9-d11b-4197-ba5d-9a93120bd293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:07.853104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebf1f9-d11b-4197-ba5d-9a93120bd293 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:07.872058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 778/1546] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:07.878715 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c6388-23ff-4bea-bbeb-f5d2730880b3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.878715 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c6388-23ff-4bea-bbeb-f5d2730880b3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.878715 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c6388-23ff-4bea-bbeb-f5d2730880b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.902685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-752c6388-23ff-4bea-bbeb-f5d2730880b3 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:07.903881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-752c6388-23ff-4bea-bbeb-f5d2730880b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:07.916711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 779/1547] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:07.918866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b402f-6b70-4002-824f-56a30fca553f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:07.918866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b402f-6b70-4002-824f-56a30fca553f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:07.918866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b402f-6b70-4002-824f-56a30fca553f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:07.927872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 769/1548] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:07 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:58:07.928592 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40379b-f8aa-446a-b351-83ba04c730e1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:07.952494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-023b402f-6b70-4002-824f-56a30fca553f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:07.953696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-023b402f-6b70-4002-824f-56a30fca553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.017578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40379b-f8aa-446a-b351-83ba04c730e1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.024446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585af73-ee77-4d51-9899-45ddc57bbec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.026039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585af73-ee77-4d51-9899-45ddc57bbec3 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.026453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585af73-ee77-4d51-9899-45ddc57bbec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.026741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585af73-ee77-4d51-9899-45ddc57bbec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.028240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585af73-ee77-4d51-9899-45ddc57bbec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.030353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585af73-ee77-4d51-9899-45ddc57bbec3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.034969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 780/1549] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.036412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4732a-0894-4d14-a22f-abc828b16aff None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.036768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4732a-0894-4d14-a22f-abc828b16aff None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.037119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4732a-0894-4d14-a22f-abc828b16aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.065423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-38f4732a-0894-4d14-a22f-abc828b16aff None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:08.065423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-38f4732a-0894-4d14-a22f-abc828b16aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.068999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 781/1550] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.070048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4732a-0894-4d14-a22f-abc828b16aff None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.079360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4732a-0894-4d14-a22f-abc828b16aff None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.083875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9f765-ce06-4d46-9443-834906ea763e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.084457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9f765-ce06-4d46-9443-834906ea763e admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.084737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9f765-ce06-4d46-9443-834906ea763e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.085011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9f765-ce06-4d46-9443-834906ea763e admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.085681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9f765-ce06-4d46-9443-834906ea763e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.087597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9f765-ce06-4d46-9443-834906ea763e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.142015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 770/1551] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:07 2022] POST /identity/v3/projects => generated 393 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:08.143449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763975-7650-4599-99e7-bef248cf9f3f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.143746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763975-7650-4599-99e7-bef248cf9f3f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.144029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4763975-7650-4599-99e7-bef248cf9f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.161787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e4763975-7650-4599-99e7-bef248cf9f3f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:08.163297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e4763975-7650-4599-99e7-bef248cf9f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.222970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 771/1552] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.224315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1513b-d720-4b7d-b7b6-a4c400525a32 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.224614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1513b-d720-4b7d-b7b6-a4c400525a32 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.224895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1513b-d720-4b7d-b7b6-a4c400525a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.242787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-85b1513b-d720-4b7d-b7b6-a4c400525a32 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:08.245738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-85b1513b-d720-4b7d-b7b6-a4c400525a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.245857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 782/1553] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:08.247613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9f765-ce06-4d46-9443-834906ea763e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.253960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 772/1554] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.255238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb2721-f769-4ba0-97e6-31e7d087b569 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.255580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb2721-f769-4ba0-97e6-31e7d087b569 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.255863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb2721-f769-4ba0-97e6-31e7d087b569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.258710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9f765-ce06-4d46-9443-834906ea763e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.264105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4783a-f50f-4732-9d23-cf913009b87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.264841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4783a-f50f-4732-9d23-cf913009b87f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.265149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4783a-f50f-4732-9d23-cf913009b87f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.265425 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4783a-f50f-4732-9d23-cf913009b87f admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.266192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4783a-f50f-4732-9d23-cf913009b87f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:08.266497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4783a-f50f-4732-9d23-cf913009b87f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.268139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4783a-f50f-4732-9d23-cf913009b87f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.275185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b8fb2721-f769-4ba0-97e6-31e7d087b569 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:08.276384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b8fb2721-f769-4ba0-97e6-31e7d087b569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.284336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 773/1555] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.285543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71943d4f-733c-459c-bfa7-435e1280f844 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.285831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71943d4f-733c-459c-bfa7-435e1280f844 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.286139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71943d4f-733c-459c-bfa7-435e1280f844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.304193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-71943d4f-733c-459c-bfa7-435e1280f844 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:08.305392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-71943d4f-733c-459c-bfa7-435e1280f844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.313813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 774/1556] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.314738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71943d4f-733c-459c-bfa7-435e1280f844 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.325981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71943d4f-733c-459c-bfa7-435e1280f844 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.328365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd3786-26b7-4ff0-b6f5-3abaac09c857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.329001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd3786-26b7-4ff0-b6f5-3abaac09c857 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.329291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd3786-26b7-4ff0-b6f5-3abaac09c857 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.329598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd3786-26b7-4ff0-b6f5-3abaac09c857 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.330350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd3786-26b7-4ff0-b6f5-3abaac09c857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:08.330665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd3786-26b7-4ff0-b6f5-3abaac09c857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.332267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd3786-26b7-4ff0-b6f5-3abaac09c857 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.334077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 783/1557] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:08.334782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4783a-f50f-4732-9d23-cf913009b87f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.345895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4783a-f50f-4732-9d23-cf913009b87f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.351161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 775/1558] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:08.351910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd101e-015c-4af2-85ed-54116e695c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.352682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7747f-dea3-4d2f-a821-a13f811e758a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.352797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd101e-015c-4af2-85ed-54116e695c6d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.353102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7747f-dea3-4d2f-a821-a13f811e758a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.353223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd101e-015c-4af2-85ed-54116e695c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.353514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7747f-dea3-4d2f-a821-a13f811e758a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.353617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd101e-015c-4af2-85ed-54116e695c6d admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.354371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd101e-015c-4af2-85ed-54116e695c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.356849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd101e-015c-4af2-85ed-54116e695c6d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.373311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e5b7747f-dea3-4d2f-a821-a13f811e758a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:08.374560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e5b7747f-dea3-4d2f-a821-a13f811e758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.383634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 776/1559] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.384418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7747f-dea3-4d2f-a821-a13f811e758a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.390541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:08.394294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7747f-dea3-4d2f-a821-a13f811e758a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.398856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed18c2-9c68-46a6-834f-7abb98755b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.399499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed18c2-9c68-46a6-834f-7abb98755b92 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.399772 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed18c2-9c68-46a6-834f-7abb98755b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.400046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed18c2-9c68-46a6-834f-7abb98755b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.400781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed18c2-9c68-46a6-834f-7abb98755b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.402529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed18c2-9c68-46a6-834f-7abb98755b92 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.412664 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 777/1560] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:08.413980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544dded-e1c2-4c3d-ba64-1609e7155602 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.414290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544dded-e1c2-4c3d-ba64-1609e7155602 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.414544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544dded-e1c2-4c3d-ba64-1609e7155602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.431949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6544dded-e1c2-4c3d-ba64-1609e7155602 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:08.433109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6544dded-e1c2-4c3d-ba64-1609e7155602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.443274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 778/1561] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.444513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c6fc6-6429-434e-a94e-55d6d56d90fe None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.444802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c6fc6-6429-434e-a94e-55d6d56d90fe None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.445078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c6fc6-6429-434e-a94e-55d6d56d90fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.448716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 784/1562] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:08.450011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58eb99-7bde-4577-a0b4-672399123a96 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.452257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58eb99-7bde-4577-a0b4-672399123a96 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.452257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58eb99-7bde-4577-a0b4-672399123a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.464400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b92c6fc6-6429-434e-a94e-55d6d56d90fe None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:08.465571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b92c6fc6-6429-434e-a94e-55d6d56d90fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.468826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-8e58eb99-7bde-4577-a0b4-672399123a96 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:08.470048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8e58eb99-7bde-4577-a0b4-672399123a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:08.473284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 779/1563] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.474184 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c6fc6-6429-434e-a94e-55d6d56d90fe None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.482498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 785/1564] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:08.483121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58eb99-7bde-4577-a0b4-672399123a96 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.483857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c6fc6-6429-434e-a94e-55d6d56d90fe None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.488428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e532628-c1a8-4835-a114-8e40e816b617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.489067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e532628-c1a8-4835-a114-8e40e816b617 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.489348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e532628-c1a8-4835-a114-8e40e816b617 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.489622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e532628-c1a8-4835-a114-8e40e816b617 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.490332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e532628-c1a8-4835-a114-8e40e816b617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.492223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e532628-c1a8-4835-a114-8e40e816b617 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.492361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58eb99-7bde-4577-a0b4-672399123a96 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.497544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5ed8d-4654-4449-b2da-bfc3b0d1710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.498390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5ed8d-4654-4449-b2da-bfc3b0d1710c admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.498574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5ed8d-4654-4449-b2da-bfc3b0d1710c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.498840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5ed8d-4654-4449-b2da-bfc3b0d1710c admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.499767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5ed8d-4654-4449-b2da-bfc3b0d1710c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.500781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-21f5ed8d-4654-4449-b2da-bfc3b0d1710c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:58:08.500781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jun 03 09:58:08.500781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: Jun 03 09:58:08.501978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5ed8d-4654-4449-b2da-bfc3b0d1710c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.598448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 780/1565] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:08.599348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e532628-c1a8-4835-a114-8e40e816b617 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.612862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 786/1566] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:08.612862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5ed8d-4654-4449-b2da-bfc3b0d1710c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.645561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:08.662448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e532628-c1a8-4835-a114-8e40e816b617 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.674713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5ed8d-4654-4449-b2da-bfc3b0d1710c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.680414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e53ff-1f3c-4146-9425-c4c4cc94248e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.681089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e53ff-1f3c-4146-9425-c4c4cc94248e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.681381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e53ff-1f3c-4146-9425-c4c4cc94248e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.681661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e53ff-1f3c-4146-9425-c4c4cc94248e admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.682434 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e53ff-1f3c-4146-9425-c4c4cc94248e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.684653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e53ff-1f3c-4146-9425-c4c4cc94248e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.694751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 787/1567] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:08.695572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e53ff-1f3c-4146-9425-c4c4cc94248e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.706071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e53ff-1f3c-4146-9425-c4c4cc94248e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.712212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cbc64-46bf-45e5-8b10-58eeb375c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.712907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbc64-46bf-45e5-8b10-58eeb375c525 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.713221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbc64-46bf-45e5-8b10-58eeb375c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.713475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbc64-46bf-45e5-8b10-58eeb375c525 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.717237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cbc64-46bf-45e5-8b10-58eeb375c525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:08.717237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cbc64-46bf-45e5-8b10-58eeb375c525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.717237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cbc64-46bf-45e5-8b10-58eeb375c525 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.726888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9791889-6497-4256-b81d-9a38a09b5973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.727484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9791889-6497-4256-b81d-9a38a09b5973 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.727749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9791889-6497-4256-b81d-9a38a09b5973 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.728027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9791889-6497-4256-b81d-9a38a09b5973 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.728748 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9791889-6497-4256-b81d-9a38a09b5973 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:08.728984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 788/1568] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:08.729079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9791889-6497-4256-b81d-9a38a09b5973 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.729877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cbc64-46bf-45e5-8b10-58eeb375c525 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.730833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9791889-6497-4256-b81d-9a38a09b5973 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.739833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cbc64-46bf-45e5-8b10-58eeb375c525 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.744922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14f896-1189-4f99-9fda-9b526cfcbb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.745575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14f896-1189-4f99-9fda-9b526cfcbb6b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.746400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14f896-1189-4f99-9fda-9b526cfcbb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.746806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14f896-1189-4f99-9fda-9b526cfcbb6b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.746951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 781/1569] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:08.747628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14f896-1189-4f99-9fda-9b526cfcbb6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.747764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9791889-6497-4256-b81d-9a38a09b5973 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.749222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14f896-1189-4f99-9fda-9b526cfcbb6b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.757690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9791889-6497-4256-b81d-9a38a09b5973 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.759968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 789/1570] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:08.760838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14f896-1189-4f99-9fda-9b526cfcbb6b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.763211 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee91f1-340f-4536-b5ed-89bfa3def268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.763921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee91f1-340f-4536-b5ed-89bfa3def268 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.764209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee91f1-340f-4536-b5ed-89bfa3def268 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.764488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee91f1-340f-4536-b5ed-89bfa3def268 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.765200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee91f1-340f-4536-b5ed-89bfa3def268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:08.765503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee91f1-340f-4536-b5ed-89bfa3def268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.767161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee91f1-340f-4536-b5ed-89bfa3def268 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.771132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14f896-1189-4f99-9fda-9b526cfcbb6b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.775792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a037edd-3eea-474c-a352-7df90f9d8f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.776428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a037edd-3eea-474c-a352-7df90f9d8f82 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.776718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a037edd-3eea-474c-a352-7df90f9d8f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.777006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a037edd-3eea-474c-a352-7df90f9d8f82 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.777748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a037edd-3eea-474c-a352-7df90f9d8f82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:08.778200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a037edd-3eea-474c-a352-7df90f9d8f82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.786576 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 782/1571] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:08.786734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a037edd-3eea-474c-a352-7df90f9d8f82 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.786837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee91f1-340f-4536-b5ed-89bfa3def268 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.798085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 790/1572] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:08.798085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a037edd-3eea-474c-a352-7df90f9d8f82 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.798340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee91f1-340f-4536-b5ed-89bfa3def268 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.803283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd45e5b-1db2-4109-8950-b0ae920aba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.803512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a037edd-3eea-474c-a352-7df90f9d8f82 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.803970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd45e5b-1db2-4109-8950-b0ae920aba70 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.804260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd45e5b-1db2-4109-8950-b0ae920aba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.804537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd45e5b-1db2-4109-8950-b0ae920aba70 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.805271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd45e5b-1db2-4109-8950-b0ae920aba70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.806431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-fbd45e5b-1db2-4109-8950-b0ae920aba70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:58:08.806431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jun 03 09:58:08.806431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: Jun 03 09:58:08.807640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd45e5b-1db2-4109-8950-b0ae920aba70 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.809323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff2b9-4609-4a9f-b774-774d0c4a06a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.810006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff2b9-4609-4a9f-b774-774d0c4a06a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.810301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff2b9-4609-4a9f-b774-774d0c4a06a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.810575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff2b9-4609-4a9f-b774-774d0c4a06a4 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.811259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff2b9-4609-4a9f-b774-774d0c4a06a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.813171 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff2b9-4609-4a9f-b774-774d0c4a06a4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.899311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 783/1573] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:08.900051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd45e5b-1db2-4109-8950-b0ae920aba70 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.909441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd45e5b-1db2-4109-8950-b0ae920aba70 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.914008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acec52c-82c9-4ba6-8af3-93701c1e0c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.916960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acec52c-82c9-4ba6-8af3-93701c1e0c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.916960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acec52c-82c9-4ba6-8af3-93701c1e0c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.916960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acec52c-82c9-4ba6-8af3-93701c1e0c2c admin admin] PATH_INFO: `/v3/projects/2887f7ffd32547458585734b104e66b5/users/a455c6862ccb4a988fd2ab6de3195cac/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.931348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 791/1574] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:08.931348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff2b9-4609-4a9f-b774-774d0c4a06a4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:08.931888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acec52c-82c9-4ba6-8af3-93701c1e0c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2887f7ffd32547458585734b104e66b5, user_id=a455c6862ccb4a988fd2ab6de3195cac, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.934399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acec52c-82c9-4ba6-8af3-93701c1e0c2c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:08.939790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff2b9-4609-4a9f-b774-774d0c4a06a4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:08.944323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a673eb-b96f-407f-a6cf-ea9af4cd68c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:08.944957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a673eb-b96f-407f-a6cf-ea9af4cd68c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:08.945238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a673eb-b96f-407f-a6cf-ea9af4cd68c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:08.945514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a673eb-b96f-407f-a6cf-ea9af4cd68c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:08.946224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a673eb-b96f-407f-a6cf-ea9af4cd68c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:08.951368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a673eb-b96f-407f-a6cf-ea9af4cd68c4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.033767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 784/1575] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:08 2022] PUT /identity/v3/projects/2887f7ffd32547458585734b104e66b5/users/a455c6862ccb4a988fd2ab6de3195cac/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:09.034601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acec52c-82c9-4ba6-8af3-93701c1e0c2c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.039632 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:09.059261 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 792/1576] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:08 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:09.060571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a673eb-b96f-407f-a6cf-ea9af4cd68c4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.096196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acec52c-82c9-4ba6-8af3-93701c1e0c2c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.122907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a673eb-b96f-407f-a6cf-ea9af4cd68c4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.127591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9634cf-7a36-4187-b32b-8c29e47091f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.128258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9634cf-7a36-4187-b32b-8c29e47091f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.128546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9634cf-7a36-4187-b32b-8c29e47091f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.128828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9634cf-7a36-4187-b32b-8c29e47091f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.129557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9634cf-7a36-4187-b32b-8c29e47091f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.131213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9634cf-7a36-4187-b32b-8c29e47091f5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.141242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 793/1577] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:09.142069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9634cf-7a36-4187-b32b-8c29e47091f5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.152811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12451a3d-8ae9-4106-bcf7-b175bdb90b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.153434 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9634cf-7a36-4187-b32b-8c29e47091f5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.153688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12451a3d-8ae9-4106-bcf7-b175bdb90b59 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.154014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12451a3d-8ae9-4106-bcf7-b175bdb90b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.154297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12451a3d-8ae9-4106-bcf7-b175bdb90b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.155007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12451a3d-8ae9-4106-bcf7-b175bdb90b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.156915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12451a3d-8ae9-4106-bcf7-b175bdb90b59 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.158316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd31fb-9a65-4c2a-9e7f-cb169ef5ddbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.158952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd31fb-9a65-4c2a-9e7f-cb169ef5ddbb admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.159239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd31fb-9a65-4c2a-9e7f-cb169ef5ddbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.159517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd31fb-9a65-4c2a-9e7f-cb169ef5ddbb admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.160257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd31fb-9a65-4c2a-9e7f-cb169ef5ddbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.162157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd31fb-9a65-4c2a-9e7f-cb169ef5ddbb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.283599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 785/1578] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:09 2022] POST /identity/v3/projects => generated 389 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:09.284845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12451a3d-8ae9-4106-bcf7-b175bdb90b59 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.310165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 794/1579] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:09 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:09.311050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd31fb-9a65-4c2a-9e7f-cb169ef5ddbb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.337441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:09.351053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12451a3d-8ae9-4106-bcf7-b175bdb90b59 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.373133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd31fb-9a65-4c2a-9e7f-cb169ef5ddbb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.378019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de5d0a-6fdd-4a81-ab2d-7149caf6335b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.378687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de5d0a-6fdd-4a81-ab2d-7149caf6335b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.378975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de5d0a-6fdd-4a81-ab2d-7149caf6335b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.379256 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de5d0a-6fdd-4a81-ab2d-7149caf6335b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.379986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de5d0a-6fdd-4a81-ab2d-7149caf6335b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.381627 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de5d0a-6fdd-4a81-ab2d-7149caf6335b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.392545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 795/1580] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:09.393364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:09.393513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de5d0a-6fdd-4a81-ab2d-7149caf6335b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.403588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de5d0a-6fdd-4a81-ab2d-7149caf6335b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.409245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf18058-e0f9-45da-9364-63f244fcbe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.410839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18058-e0f9-45da-9364-63f244fcbe38 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.410839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18058-e0f9-45da-9364-63f244fcbe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.410839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18058-e0f9-45da-9364-63f244fcbe38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.411116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf9cb4-aee0-4659-bd9c-3f52486816c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.411224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf18058-e0f9-45da-9364-63f244fcbe38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.411771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf9cb4-aee0-4659-bd9c-3f52486816c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.412057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf9cb4-aee0-4659-bd9c-3f52486816c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.412330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf9cb4-aee0-4659-bd9c-3f52486816c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.412774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf18058-e0f9-45da-9364-63f244fcbe38 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.413006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf9cb4-aee0-4659-bd9c-3f52486816c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.414970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf9cb4-aee0-4659-bd9c-3f52486816c2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.423280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 796/1581] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:09.424630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbee50a-8877-4d7a-8c92-77916e2241de None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.424954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbee50a-8877-4d7a-8c92-77916e2241de None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.425512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbee50a-8877-4d7a-8c92-77916e2241de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.450593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7cbee50a-8877-4d7a-8c92-77916e2241de None None] MFA Rules not processed for user `a455c6862ccb4a988fd2ab6de3195cac`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:09.451781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7cbee50a-8877-4d7a-8c92-77916e2241de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:09.494758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 786/1582] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:09 2022] POST /identity/v3/projects => generated 369 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:09.495717 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf9cb4-aee0-4659-bd9c-3f52486816c2 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.518895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 797/1583] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:09.523986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbee50a-8877-4d7a-8c92-77916e2241de None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.534971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbee50a-8877-4d7a-8c92-77916e2241de None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.560067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf9cb4-aee0-4659-bd9c-3f52486816c2 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.564719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0516778-840e-45c1-8aff-bd4c37d6d1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.565393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0516778-840e-45c1-8aff-bd4c37d6d1fb admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.565681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0516778-840e-45c1-8aff-bd4c37d6d1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.565998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0516778-840e-45c1-8aff-bd4c37d6d1fb admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.566713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0516778-840e-45c1-8aff-bd4c37d6d1fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.568678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0516778-840e-45c1-8aff-bd4c37d6d1fb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.594481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b6d66-f0d6-433e-86ce-92c17c2df402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.595396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b6d66-f0d6-433e-86ce-92c17c2df402 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.595683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b6d66-f0d6-433e-86ce-92c17c2df402 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.595963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b6d66-f0d6-433e-86ce-92c17c2df402 admin admin] PATH_INFO: `/v3/projects/071b399988474af1a1c709aac97c1744/users/6113d715500240098217caf641db763a/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.610667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b6d66-f0d6-433e-86ce-92c17c2df402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=071b399988474af1a1c709aac97c1744, user_id=6113d715500240098217caf641db763a, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.611706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-e71b6d66-f0d6-433e-86ce-92c17c2df402 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:58:09.611706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jun 03 09:58:09.611706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: Jun 03 09:58:09.613605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b6d66-f0d6-433e-86ce-92c17c2df402 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.693747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 787/1584] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:09 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:09.695104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0516778-840e-45c1-8aff-bd4c37d6d1fb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.706165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0516778-840e-45c1-8aff-bd4c37d6d1fb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.711088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea16075-bc36-400b-a3e4-ba2609d23d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.711963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea16075-bc36-400b-a3e4-ba2609d23d6d admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.712310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea16075-bc36-400b-a3e4-ba2609d23d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.712647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea16075-bc36-400b-a3e4-ba2609d23d6d admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.713412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea16075-bc36-400b-a3e4-ba2609d23d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.717400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea16075-bc36-400b-a3e4-ba2609d23d6d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.725164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 798/1585] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:09 2022] PUT /identity/v3/projects/071b399988474af1a1c709aac97c1744/users/6113d715500240098217caf641db763a/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:09.727394 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b6d66-f0d6-433e-86ce-92c17c2df402 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.752625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:09.789111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b6d66-f0d6-433e-86ce-92c17c2df402 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.793765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ae269-bfb0-4f99-ad5e-18ff7a6c854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.794485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ae269-bfb0-4f99-ad5e-18ff7a6c854a admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.794776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ae269-bfb0-4f99-ad5e-18ff7a6c854a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.795073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ae269-bfb0-4f99-ad5e-18ff7a6c854a admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.795779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ae269-bfb0-4f99-ad5e-18ff7a6c854a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.797728 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ae269-bfb0-4f99-ad5e-18ff7a6c854a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.815593 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 788/1586] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:09 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:09.816409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea16075-bc36-400b-a3e4-ba2609d23d6d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.826435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea16075-bc36-400b-a3e4-ba2609d23d6d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.830923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882726b-ebcf-4c2c-a630-a43f21dea9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.831555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882726b-ebcf-4c2c-a630-a43f21dea9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.831837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882726b-ebcf-4c2c-a630-a43f21dea9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.832114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882726b-ebcf-4c2c-a630-a43f21dea9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.832872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882726b-ebcf-4c2c-a630-a43f21dea9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.834511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882726b-ebcf-4c2c-a630-a43f21dea9a2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.845622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 789/1587] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:09.846436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882726b-ebcf-4c2c-a630-a43f21dea9a2 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.856593 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882726b-ebcf-4c2c-a630-a43f21dea9a2 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.861224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f86d2-f9e4-4240-9717-c5935e5fdd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.861854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f86d2-f9e4-4240-9717-c5935e5fdd2f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.862172 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f86d2-f9e4-4240-9717-c5935e5fdd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.862453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f86d2-f9e4-4240-9717-c5935e5fdd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.863145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f86d2-f9e4-4240-9717-c5935e5fdd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.864674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f86d2-f9e4-4240-9717-c5935e5fdd2f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.874416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 790/1588] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:09.875202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f86d2-f9e4-4240-9717-c5935e5fdd2f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.884214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f86d2-f9e4-4240-9717-c5935e5fdd2f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.890768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef4354-a65f-4069-b796-5b066368cd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.891431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef4354-a65f-4069-b796-5b066368cd88 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.892018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef4354-a65f-4069-b796-5b066368cd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.892305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef4354-a65f-4069-b796-5b066368cd88 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.893264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef4354-a65f-4069-b796-5b066368cd88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.895082 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 799/1589] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:09 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:09.895735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef4354-a65f-4069-b796-5b066368cd88 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.896473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a14e1-6ef6-42ec-8226-30f59b885e3f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.896815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a14e1-6ef6-42ec-8226-30f59b885e3f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.897035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a14e1-6ef6-42ec-8226-30f59b885e3f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.898328 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 800/1590] 10.222.0.42 () {54 vars in 961 bytes} [Fri Jun 3 09:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:58:09.903912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a14e1-6ef6-42ec-8226-30f59b885e3f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.916661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a14e1-6ef6-42ec-8226-30f59b885e3f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.921140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a98c57-96e6-4273-b044-a37f017eb533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.921785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a98c57-96e6-4273-b044-a37f017eb533 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.922108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a98c57-96e6-4273-b044-a37f017eb533 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.922390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a98c57-96e6-4273-b044-a37f017eb533 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.923107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a98c57-96e6-4273-b044-a37f017eb533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.924721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a98c57-96e6-4273-b044-a37f017eb533 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.934461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 801/1591] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:09.935613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87243ed-5651-46b3-a8fe-fd3f858e37b1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.935933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87243ed-5651-46b3-a8fe-fd3f858e37b1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.936211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87243ed-5651-46b3-a8fe-fd3f858e37b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.953383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e87243ed-5651-46b3-a8fe-fd3f858e37b1 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:09.954535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e87243ed-5651-46b3-a8fe-fd3f858e37b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:09.962293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 802/1592] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:09.963043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87243ed-5651-46b3-a8fe-fd3f858e37b1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.973464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87243ed-5651-46b3-a8fe-fd3f858e37b1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.978976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 791/1593] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:09 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:09.978976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef4354-a65f-4069-b796-5b066368cd88 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:09.979220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe2f6a-2832-4116-8358-57c09bfaacd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.979576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe2f6a-2832-4116-8358-57c09bfaacd2 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.980089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe2f6a-2832-4116-8358-57c09bfaacd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.981316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe2f6a-2832-4116-8358-57c09bfaacd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.982508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe2f6a-2832-4116-8358-57c09bfaacd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:09.984396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe2f6a-2832-4116-8358-57c09bfaacd2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:09.987394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef4354-a65f-4069-b796-5b066368cd88 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:09.992798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f9838-91d4-45d2-8b49-bff949f9c6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:09.993704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f9838-91d4-45d2-8b49-bff949f9c6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:09.994038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f9838-91d4-45d2-8b49-bff949f9c6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:09.994325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f9838-91d4-45d2-8b49-bff949f9c6ec admin admin] PATH_INFO: `/v3/projects/686f552be5824c8aa31930a6e8844255/users/49efae911ef8458b9288558014217934/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:09.994771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 803/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:09.995486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe2f6a-2832-4116-8358-57c09bfaacd2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.005397 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe2f6a-2832-4116-8358-57c09bfaacd2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.010251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa383eeb-ee10-4291-85f7-c08f1c882702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.012896 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f9838-91d4-45d2-8b49-bff949f9c6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=686f552be5824c8aa31930a6e8844255, user_id=49efae911ef8458b9288558014217934, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.013063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa383eeb-ee10-4291-85f7-c08f1c882702 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.013063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa383eeb-ee10-4291-85f7-c08f1c882702 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.013063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa383eeb-ee10-4291-85f7-c08f1c882702 admin admin] PATH_INFO: `/v3/projects/69776fa5af0c4573b668bbf8dc200931/users/d2ed7dd8d03c4c42925e3efca1915a46/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.013853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f9838-91d4-45d2-8b49-bff949f9c6ec admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.026447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa383eeb-ee10-4291-85f7-c08f1c882702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69776fa5af0c4573b668bbf8dc200931, user_id=d2ed7dd8d03c4c42925e3efca1915a46, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.029134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa383eeb-ee10-4291-85f7-c08f1c882702 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.116900 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 792/1595] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:09 2022] PUT /identity/v3/projects/686f552be5824c8aa31930a6e8844255/users/49efae911ef8458b9288558014217934/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:10.117912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f9838-91d4-45d2-8b49-bff949f9c6ec admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.150795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 804/1596] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:09 2022] PUT /identity/v3/projects/69776fa5af0c4573b668bbf8dc200931/users/d2ed7dd8d03c4c42925e3efca1915a46/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:10.152653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e77427-0c5c-4556-a229-e24a6bcb21cc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.155941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e77427-0c5c-4556-a229-e24a6bcb21cc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.155941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e77427-0c5c-4556-a229-e24a6bcb21cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.159654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:10.171226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e8e77427-0c5c-4556-a229-e24a6bcb21cc None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:10.172436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e8e77427-0c5c-4556-a229-e24a6bcb21cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:10.180777 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f9838-91d4-45d2-8b49-bff949f9c6ec admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.236742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439f088-2c53-4b98-b0f2-d9751741072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.237552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439f088-2c53-4b98-b0f2-d9751741072e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.237839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439f088-2c53-4b98-b0f2-d9751741072e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.238146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439f088-2c53-4b98-b0f2-d9751741072e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.238870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439f088-2c53-4b98-b0f2-d9751741072e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.240616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439f088-2c53-4b98-b0f2-d9751741072e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.242035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 805/1597] 10.222.0.42 () {58 vars in 1058 bytes} [Fri Jun 3 09:58:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:10.242902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e77427-0c5c-4556-a229-e24a6bcb21cc None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.252659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 793/1598] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:10.253540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439f088-2c53-4b98-b0f2-d9751741072e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.255285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0439f088-2c53-4b98-b0f2-d9751741072e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:10.259817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e77427-0c5c-4556-a229-e24a6bcb21cc None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.264882 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb147-5c29-41df-bb2c-841f9a2cd9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.265595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb147-5c29-41df-bb2c-841f9a2cd9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.265878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb147-5c29-41df-bb2c-841f9a2cd9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.266183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fb147-5c29-41df-bb2c-841f9a2cd9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.266912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fb147-5c29-41df-bb2c-841f9a2cd9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.268681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fb147-5c29-41df-bb2c-841f9a2cd9e7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.279310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 806/1599] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:10.283190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb147-5c29-41df-bb2c-841f9a2cd9e7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.292674 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fb147-5c29-41df-bb2c-841f9a2cd9e7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.297915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2833c-abd3-49e4-82b9-7b0bc2b61f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.298564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2833c-abd3-49e4-82b9-7b0bc2b61f85 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.298854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2833c-abd3-49e4-82b9-7b0bc2b61f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.299134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2833c-abd3-49e4-82b9-7b0bc2b61f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.299933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2833c-abd3-49e4-82b9-7b0bc2b61f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.301505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2833c-abd3-49e4-82b9-7b0bc2b61f85 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.311516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 807/1600] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:10.312237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2833c-abd3-49e4-82b9-7b0bc2b61f85 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.313763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439f088-2c53-4b98-b0f2-d9751741072e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.317497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a69cc4-007e-444f-bcf9-66589b6636f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.318185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a69cc4-007e-444f-bcf9-66589b6636f0 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.318471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a69cc4-007e-444f-bcf9-66589b6636f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.318752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a69cc4-007e-444f-bcf9-66589b6636f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.321822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2833c-abd3-49e4-82b9-7b0bc2b61f85 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.327859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353ea35-b34c-46a7-8480-41b15ddf5fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.328537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ea35-b34c-46a7-8480-41b15ddf5fbd admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.328778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ea35-b34c-46a7-8480-41b15ddf5fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.329027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a69cc4-007e-444f-bcf9-66589b6636f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.329142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ea35-b34c-46a7-8480-41b15ddf5fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.329751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353ea35-b34c-46a7-8480-41b15ddf5fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.330518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a69cc4-007e-444f-bcf9-66589b6636f0 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.331348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353ea35-b34c-46a7-8480-41b15ddf5fbd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.341175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 794/1601] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:10.342345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde5c80-e39a-4543-8bb7-5fc4cae28388 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.345996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde5c80-e39a-4543-8bb7-5fc4cae28388 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.346156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 808/1602] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:10.346267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde5c80-e39a-4543-8bb7-5fc4cae28388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.346370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044598-e780-413c-bd9a-0490fdd8a722 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.346370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044598-e780-413c-bd9a-0490fdd8a722 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.346370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1044598-e780-413c-bd9a-0490fdd8a722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.362439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d1044598-e780-413c-bd9a-0490fdd8a722 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:10.362663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5fde5c80-e39a-4543-8bb7-5fc4cae28388 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:10.363799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d1044598-e780-413c-bd9a-0490fdd8a722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:10.363931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5fde5c80-e39a-4543-8bb7-5fc4cae28388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:10.371861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 809/1603] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:10.372576 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 795/1604] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:10.372783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1044598-e780-413c-bd9a-0490fdd8a722 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.374815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde5c80-e39a-4543-8bb7-5fc4cae28388 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.389368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde5c80-e39a-4543-8bb7-5fc4cae28388 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.389985 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1044598-e780-413c-bd9a-0490fdd8a722 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.394094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23f07a-568b-468b-9e00-4d6bcdabfe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.394858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f07a-568b-468b-9e00-4d6bcdabfe0a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.395198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f07a-568b-468b-9e00-4d6bcdabfe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.395481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23f07a-568b-468b-9e00-4d6bcdabfe0a admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.395928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c19d2-db40-411d-a2f6-ccc39eebcad1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.396210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23f07a-568b-468b-9e00-4d6bcdabfe0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.396575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c19d2-db40-411d-a2f6-ccc39eebcad1 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.396858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c19d2-db40-411d-a2f6-ccc39eebcad1 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.397134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c19d2-db40-411d-a2f6-ccc39eebcad1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.397824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23f07a-568b-468b-9e00-4d6bcdabfe0a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.406463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:10.408909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 796/1605] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:10.412256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23f07a-568b-468b-9e00-4d6bcdabfe0a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.419491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23f07a-568b-468b-9e00-4d6bcdabfe0a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.424201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0802d51-e16b-40c5-bf5f-af733098cc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.425063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0802d51-e16b-40c5-bf5f-af733098cc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.425349 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0802d51-e16b-40c5-bf5f-af733098cc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.425625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0802d51-e16b-40c5-bf5f-af733098cc70 admin admin] PATH_INFO: `/v3/projects/3c94df3549d74809933492001a652f2b/users/1d7675d5d55e43f7940c9ea5b213b559/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.443562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0802d51-e16b-40c5-bf5f-af733098cc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c94df3549d74809933492001a652f2b, user_id=1d7675d5d55e43f7940c9ea5b213b559, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.446100 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0802d51-e16b-40c5-bf5f-af733098cc70 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.457548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c19d2-db40-411d-a2f6-ccc39eebcad1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.459175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c19d2-db40-411d-a2f6-ccc39eebcad1 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.468191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 810/1606] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:10.468979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c19d2-db40-411d-a2f6-ccc39eebcad1 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.479464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c19d2-db40-411d-a2f6-ccc39eebcad1 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.484540 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fecbbd-00f4-4a34-8aaa-bb180d661f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.489399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fecbbd-00f4-4a34-8aaa-bb180d661f15 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.489399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fecbbd-00f4-4a34-8aaa-bb180d661f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.489399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fecbbd-00f4-4a34-8aaa-bb180d661f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.489399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fecbbd-00f4-4a34-8aaa-bb180d661f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.489399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fecbbd-00f4-4a34-8aaa-bb180d661f15 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.498875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 811/1607] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:10.502795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fecbbd-00f4-4a34-8aaa-bb180d661f15 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.513111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fecbbd-00f4-4a34-8aaa-bb180d661f15 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.521306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb169f4-bf75-496d-bfdf-b135e7fa334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.523799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb169f4-bf75-496d-bfdf-b135e7fa334e admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.523799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb169f4-bf75-496d-bfdf-b135e7fa334e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.523799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb169f4-bf75-496d-bfdf-b135e7fa334e admin admin] PATH_INFO: `/v3/projects/57756244f164417dad785bb5927d05b6/users/5a304d58eff84a87953dc7af6a0afe87/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.539154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb169f4-bf75-496d-bfdf-b135e7fa334e admin admin] RBAC: Authorizing `identity:create_grant(project_id=57756244f164417dad785bb5927d05b6, user_id=5a304d58eff84a87953dc7af6a0afe87, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.541875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb169f4-bf75-496d-bfdf-b135e7fa334e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.558142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 797/1608] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:10 2022] PUT /identity/v3/projects/3c94df3549d74809933492001a652f2b/users/1d7675d5d55e43f7940c9ea5b213b559/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:10.559923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0802d51-e16b-40c5-bf5f-af733098cc70 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.605159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:10.615876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0802d51-e16b-40c5-bf5f-af733098cc70 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.619541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e2e73-a088-4a5c-b3d3-a6cfe8c9054c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.620181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e2e73-a088-4a5c-b3d3-a6cfe8c9054c service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.620465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e2e73-a088-4a5c-b3d3-a6cfe8c9054c service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.620743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e2e73-a088-4a5c-b3d3-a6cfe8c9054c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.654277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 812/1609] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:10 2022] PUT /identity/v3/projects/57756244f164417dad785bb5927d05b6/users/5a304d58eff84a87953dc7af6a0afe87/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:10.658441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb169f4-bf75-496d-bfdf-b135e7fa334e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.725662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e2e73-a088-4a5c-b3d3-a6cfe8c9054c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.727368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e2e73-a088-4a5c-b3d3-a6cfe8c9054c service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.733956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb169f4-bf75-496d-bfdf-b135e7fa334e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.737774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e562577-9cf3-443b-8f15-a9c0bf6b1c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.738568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e562577-9cf3-443b-8f15-a9c0bf6b1c30 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.749029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e562577-9cf3-443b-8f15-a9c0bf6b1c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.749029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e562577-9cf3-443b-8f15-a9c0bf6b1c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.749356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 798/1610] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:10.749356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e2e73-a088-4a5c-b3d3-a6cfe8c9054c service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.750706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e2e73-a088-4a5c-b3d3-a6cfe8c9054c service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.817180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22115f0c-0558-4b19-8593-e5b5defe682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.817546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e562577-9cf3-443b-8f15-a9c0bf6b1c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.818208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115f0c-0558-4b19-8593-e5b5defe682c admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.818493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115f0c-0558-4b19-8593-e5b5defe682c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.818825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115f0c-0558-4b19-8593-e5b5defe682c admin admin] PATH_INFO: `/v3/projects/b6522d9aaf5a45b2910df87d13a0b30d/users/7031f44ed4c64aa8b8e236cb8e3c1b9e/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.819721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e562577-9cf3-443b-8f15-a9c0bf6b1c30 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.829899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 813/1611] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:10.830853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e562577-9cf3-443b-8f15-a9c0bf6b1c30 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:10.834926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22115f0c-0558-4b19-8593-e5b5defe682c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6522d9aaf5a45b2910df87d13a0b30d, user_id=7031f44ed4c64aa8b8e236cb8e3c1b9e, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.837212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22115f0c-0558-4b19-8593-e5b5defe682c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.841280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e562577-9cf3-443b-8f15-a9c0bf6b1c30 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:10.845525 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff753382-3f97-4e8b-acac-05724c97f09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:10.846467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff753382-3f97-4e8b-acac-05724c97f09f admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.846739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff753382-3f97-4e8b-acac-05724c97f09f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.847009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff753382-3f97-4e8b-acac-05724c97f09f admin admin] PATH_INFO: `/v3/projects/b4991062b0ee46f3a1be8482be7ef5df/users/e5c1bbcf86214f8c896aa6d67203f0f9/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.862678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff753382-3f97-4e8b-acac-05724c97f09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4991062b0ee46f3a1be8482be7ef5df, user_id=e5c1bbcf86214f8c896aa6d67203f0f9, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:10.865340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff753382-3f97-4e8b-acac-05724c97f09f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:10.919672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 799/1612] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:10 2022] PUT /identity/v3/projects/b6522d9aaf5a45b2910df87d13a0b30d/users/7031f44ed4c64aa8b8e236cb8e3c1b9e/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:10.919672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6644f4a-4c78-4385-a744-b6f4547d4194 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.919672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6644f4a-4c78-4385-a744-b6f4547d4194 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.919672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6644f4a-4c78-4385-a744-b6f4547d4194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.951748 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b6644f4a-4c78-4385-a744-b6f4547d4194 None None] MFA Rules not processed for user `1d7675d5d55e43f7940c9ea5b213b559`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:10.953723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b6644f4a-4c78-4385-a744-b6f4547d4194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:10.956403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:10.964467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 814/1613] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:10 2022] PUT /identity/v3/projects/b4991062b0ee46f3a1be8482be7ef5df/users/e5c1bbcf86214f8c896aa6d67203f0f9/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:10.967210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf04de-2b5a-43c5-9eff-c41771666aa5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:10.967863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf04de-2b5a-43c5-9eff-c41771666aa5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:10.968573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf04de-2b5a-43c5-9eff-c41771666aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:10.990908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-19bf04de-2b5a-43c5-9eff-c41771666aa5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:10.993720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-19bf04de-2b5a-43c5-9eff-c41771666aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:11.076016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 815/1614] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:11.077589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435dc9b-aa48-42aa-acb9-d8c239bd68a4 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.077799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435dc9b-aa48-42aa-acb9-d8c239bd68a4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.078110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435dc9b-aa48-42aa-acb9-d8c239bd68a4 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.079828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 816/1615] 10.222.0.42 () {54 vars in 912 bytes} [Fri Jun 3 09:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:58:11.081870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0628e6c-d52e-4642-ba15-366e9dbe1d40 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.082191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0628e6c-d52e-4642-ba15-366e9dbe1d40 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.082481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0628e6c-d52e-4642-ba15-366e9dbe1d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.092230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 800/1616] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:11.093573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6fd6e-b210-49c2-970c-68e9501dbd8b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.094086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6fd6e-b210-49c2-970c-68e9501dbd8b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.094446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6fd6e-b210-49c2-970c-68e9501dbd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.101492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:11.111931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a0628e6c-d52e-4642-ba15-366e9dbe1d40 None None] MFA Rules not processed for user `7031f44ed4c64aa8b8e236cb8e3c1b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:11.113223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a0628e6c-d52e-4642-ba15-366e9dbe1d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:11.136201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4cd6fd6e-b210-49c2-970c-68e9501dbd8b None None] MFA Rules not processed for user `e5c1bbcf86214f8c896aa6d67203f0f9`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:11.137996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4cd6fd6e-b210-49c2-970c-68e9501dbd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:11.206813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 817/1617] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:11.208226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5566c-afae-429a-91a3-cd3ec3b3f137 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.208588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5566c-afae-429a-91a3-cd3ec3b3f137 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.208942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5566c-afae-429a-91a3-cd3ec3b3f137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.234194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a0c5566c-afae-429a-91a3-cd3ec3b3f137 None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:11.234194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a0c5566c-afae-429a-91a3-cd3ec3b3f137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:11.250686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 801/1618] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:11.251724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6fd6e-b210-49c2-970c-68e9501dbd8b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.319212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 818/1619] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 09:58:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:11.320053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5566c-afae-429a-91a3-cd3ec3b3f137 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.334915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6fd6e-b210-49c2-970c-68e9501dbd8b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:11.341918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c210f-ac1e-4a5d-ae8d-c2d4b955d28f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:11.342439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c210f-ac1e-4a5d-ae8d-c2d4b955d28f service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.342788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c210f-ac1e-4a5d-ae8d-c2d4b955d28f service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.343621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c210f-ac1e-4a5d-ae8d-c2d4b955d28f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.349671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5566c-afae-429a-91a3-cd3ec3b3f137 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:11.354831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c210f-ac1e-4a5d-ae8d-c2d4b955d28f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:11.361801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98d027-c926-4cb2-8d7b-53f75be825c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:11.362212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c210f-ac1e-4a5d-ae8d-c2d4b955d28f service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:11.362311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98d027-c926-4cb2-8d7b-53f75be825c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.362311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98d027-c926-4cb2-8d7b-53f75be825c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.362311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98d027-c926-4cb2-8d7b-53f75be825c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.362311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98d027-c926-4cb2-8d7b-53f75be825c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:11.362949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98d027-c926-4cb2-8d7b-53f75be825c2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:11.366560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 802/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:11.367562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c210f-ac1e-4a5d-ae8d-c2d4b955d28f service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.377414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c210f-ac1e-4a5d-ae8d-c2d4b955d28f service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:11.391433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58416c-276f-40f8-8e83-5271b8e7c08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:11.392641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58416c-276f-40f8-8e83-5271b8e7c08d admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.393145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58416c-276f-40f8-8e83-5271b8e7c08d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.393517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58416c-276f-40f8-8e83-5271b8e7c08d admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.395902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58416c-276f-40f8-8e83-5271b8e7c08d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:11.398027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58416c-276f-40f8-8e83-5271b8e7c08d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:11.493544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:11.500684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 803/1621] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:11 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:11.502491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9021da8-e398-4fdd-bcc9-b42c818e3e6c None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.502789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9021da8-e398-4fdd-bcc9-b42c818e3e6c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.503655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9021da8-e398-4fdd-bcc9-b42c818e3e6c None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.505508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 819/1622] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:11 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:11.506701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 804/1623] 10.222.0.42 () {54 vars in 961 bytes} [Fri Jun 3 09:58:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:58:11.507495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9021da8-e398-4fdd-bcc9-b42c818e3e6c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.508029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98d027-c926-4cb2-8d7b-53f75be825c2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.579347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98d027-c926-4cb2-8d7b-53f75be825c2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:11.583997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7406508-9f4e-44ae-8800-5505bc22322c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:11.587524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7406508-9f4e-44ae-8800-5505bc22322c service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.587524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7406508-9f4e-44ae-8800-5505bc22322c service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.587524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7406508-9f4e-44ae-8800-5505bc22322c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.587810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9021da8-e398-4fdd-bcc9-b42c818e3e6c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:11.592329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae300db9-4208-44aa-a2f7-1052c72d19ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:11.592552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae300db9-4208-44aa-a2f7-1052c72d19ad admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.592938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae300db9-4208-44aa-a2f7-1052c72d19ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.593260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae300db9-4208-44aa-a2f7-1052c72d19ad admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.594058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae300db9-4208-44aa-a2f7-1052c72d19ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:11.596173 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae300db9-4208-44aa-a2f7-1052c72d19ad admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:11.690529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7406508-9f4e-44ae-8800-5505bc22322c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:11.705109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 805/1624] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:11 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:11.706293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae300db9-4208-44aa-a2f7-1052c72d19ad admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.710997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7406508-9f4e-44ae-8800-5505bc22322c service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:11.725531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:11.734473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae300db9-4208-44aa-a2f7-1052c72d19ad admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:11.743557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 820/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:11.745265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118f9ea-de0c-4c40-afb0-e6fcdd25118f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.747086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118f9ea-de0c-4c40-afb0-e6fcdd25118f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.747367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118f9ea-de0c-4c40-afb0-e6fcdd25118f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.748215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba4018-235f-462a-becf-0ffe5a4a6e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:11.748934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba4018-235f-462a-becf-0ffe5a4a6e13 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.749212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba4018-235f-462a-becf-0ffe5a4a6e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.749479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba4018-235f-462a-becf-0ffe5a4a6e13 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.752146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba4018-235f-462a-becf-0ffe5a4a6e13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:11.756285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba4018-235f-462a-becf-0ffe5a4a6e13 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:11.787539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1118f9ea-de0c-4c40-afb0-e6fcdd25118f None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:11.788826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1118f9ea-de0c-4c40-afb0-e6fcdd25118f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:11.805637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 821/1626] 10.222.0.42 () {58 vars in 1058 bytes} [Fri Jun 3 09:58:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:11.805637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118f9ea-de0c-4c40-afb0-e6fcdd25118f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.813424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118f9ea-de0c-4c40-afb0-e6fcdd25118f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:11.820000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30e907-bc2d-4296-bc7f-f4afd8c30fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:11.821003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30e907-bc2d-4296-bc7f-f4afd8c30fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.821618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30e907-bc2d-4296-bc7f-f4afd8c30fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.822217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30e907-bc2d-4296-bc7f-f4afd8c30fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.823183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30e907-bc2d-4296-bc7f-f4afd8c30fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:11.828136 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30e907-bc2d-4296-bc7f-f4afd8c30fa5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:11.845928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 822/1627] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:11.845928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30e907-bc2d-4296-bc7f-f4afd8c30fa5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.857192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30e907-bc2d-4296-bc7f-f4afd8c30fa5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:11.863651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cf4fa-7422-4613-85a6-25bf1b38644f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:11.864393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cf4fa-7422-4613-85a6-25bf1b38644f service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.864690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cf4fa-7422-4613-85a6-25bf1b38644f service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.865567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cf4fa-7422-4613-85a6-25bf1b38644f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.908762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 806/1628] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:11 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:11.908762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba4018-235f-462a-becf-0ffe5a4a6e13 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.928604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba4018-235f-462a-becf-0ffe5a4a6e13 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:11.938653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cf4fa-7422-4613-85a6-25bf1b38644f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:11.941177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cf4fa-7422-4613-85a6-25bf1b38644f service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:11.966590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3def5-121b-46d2-96a8-3ad378d376bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:11.967583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3def5-121b-46d2-96a8-3ad378d376bd admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:11.967796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3def5-121b-46d2-96a8-3ad378d376bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:11.968324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3def5-121b-46d2-96a8-3ad378d376bd admin admin] PATH_INFO: `/v3/projects/6c11f9aab8664daa8ea171cf44ccd571/users/9e5eadb19da7460bb7584d213c5b216b/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:11.971000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 823/1629] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:11.976908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cf4fa-7422-4613-85a6-25bf1b38644f service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:11.998173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cf4fa-7422-4613-85a6-25bf1b38644f service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:12.009158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c238d-bfa0-4c31-9fdd-b9c022ac1c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:12.010688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c238d-bfa0-4c31-9fdd-b9c022ac1c62 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.012470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c238d-bfa0-4c31-9fdd-b9c022ac1c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.013166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c238d-bfa0-4c31-9fdd-b9c022ac1c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.015995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c238d-bfa0-4c31-9fdd-b9c022ac1c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:12.019963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c238d-bfa0-4c31-9fdd-b9c022ac1c62 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:12.031895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3def5-121b-46d2-96a8-3ad378d376bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c11f9aab8664daa8ea171cf44ccd571, user_id=9e5eadb19da7460bb7584d213c5b216b, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:12.039653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3def5-121b-46d2-96a8-3ad378d376bd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:12.045900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 824/1630] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:12.046950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c238d-bfa0-4c31-9fdd-b9c022ac1c62 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:12.070410 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c238d-bfa0-4c31-9fdd-b9c022ac1c62 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:12.085340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9de35-8d6f-4c21-923c-e058d5695fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:12.086466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9de35-8d6f-4c21-923c-e058d5695fac admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.087283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9de35-8d6f-4c21-923c-e058d5695fac admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.090279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9de35-8d6f-4c21-923c-e058d5695fac admin admin] PATH_INFO: `/v3/projects/70fa0dc41e80462a9ba09e4e44ee509a/users/46a5ece1d2f443708aa542a62100dfcd/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.131421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9de35-8d6f-4c21-923c-e058d5695fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fa0dc41e80462a9ba09e4e44ee509a, user_id=46a5ece1d2f443708aa542a62100dfcd, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:12.134598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9de35-8d6f-4c21-923c-e058d5695fac admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:12.149471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 807/1631] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:11 2022] PUT /identity/v3/projects/6c11f9aab8664daa8ea171cf44ccd571/users/9e5eadb19da7460bb7584d213c5b216b/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:12.150709 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3def5-121b-46d2-96a8-3ad378d376bd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:12.216409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:12.254636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 825/1632] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:12 2022] PUT /identity/v3/projects/70fa0dc41e80462a9ba09e4e44ee509a/users/46a5ece1d2f443708aa542a62100dfcd/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:12.262056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9de35-8d6f-4c21-923c-e058d5695fac admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:12.272058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3def5-121b-46d2-96a8-3ad378d376bd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:12.368125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64f9b1-bf18-4fef-9b71-6e303a7c65bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:12.369870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64f9b1-bf18-4fef-9b71-6e303a7c65bc admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.370429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64f9b1-bf18-4fef-9b71-6e303a7c65bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.370939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64f9b1-bf18-4fef-9b71-6e303a7c65bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.371756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64f9b1-bf18-4fef-9b71-6e303a7c65bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:12.374781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64f9b1-bf18-4fef-9b71-6e303a7c65bc admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:12.399564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 808/1633] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:12.404044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2e59-d731-40f6-8e38-1d5e62435864 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.406655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2e59-d731-40f6-8e38-1d5e62435864 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.407064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f2e59-d731-40f6-8e38-1d5e62435864 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.411928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 809/1634] 10.222.0.42 () {54 vars in 912 bytes} [Fri Jun 3 09:58:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:58:12.414559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2e59-d731-40f6-8e38-1d5e62435864 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:12.424357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9de35-8d6f-4c21-923c-e058d5695fac admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:12.436608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144db3b8-4506-41db-a723-12a0955800bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:12.440648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144db3b8-4506-41db-a723-12a0955800bc admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.440648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144db3b8-4506-41db-a723-12a0955800bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.440648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144db3b8-4506-41db-a723-12a0955800bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.440648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144db3b8-4506-41db-a723-12a0955800bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:12.440648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144db3b8-4506-41db-a723-12a0955800bc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:12.450340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f2e59-d731-40f6-8e38-1d5e62435864 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:12.463768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a03a8-b146-4e82-9a14-85dc95c83dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:12.465691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a03a8-b146-4e82-9a14-85dc95c83dda admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.466244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a03a8-b146-4e82-9a14-85dc95c83dda admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.466533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a03a8-b146-4e82-9a14-85dc95c83dda admin admin] PATH_INFO: `/v3/projects/6c11f9aab8664daa8ea171cf44ccd571/users/9e5eadb19da7460bb7584d213c5b216b/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.468414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 826/1635] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:12.469022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a03a8-b146-4e82-9a14-85dc95c83dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c11f9aab8664daa8ea171cf44ccd571, user_id=9e5eadb19da7460bb7584d213c5b216b, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:12.469861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f533221-12a3-4717-b0c7-908867641f42 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.471127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f533221-12a3-4717-b0c7-908867641f42 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.471127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f533221-12a3-4717-b0c7-908867641f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.473285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a03a8-b146-4e82-9a14-85dc95c83dda admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:12.489938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 810/1636] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:12 2022] PUT /identity/v3/projects/6c11f9aab8664daa8ea171cf44ccd571/users/9e5eadb19da7460bb7584d213c5b216b/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:12.504106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a03a8-b146-4e82-9a14-85dc95c83dda admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:12.535068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9f533221-12a3-4717-b0c7-908867641f42 None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:12.543293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9f533221-12a3-4717-b0c7-908867641f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:12.627679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:12.641268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a03a8-b146-4e82-9a14-85dc95c83dda admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:12.647529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54ecd2-96f3-4940-a097-5429887b6d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:12.648550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54ecd2-96f3-4940-a097-5429887b6d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.649187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54ecd2-96f3-4940-a097-5429887b6d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.649707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54ecd2-96f3-4940-a097-5429887b6d5a admin admin] PATH_INFO: `/v3/projects/70fa0dc41e80462a9ba09e4e44ee509a/users/46a5ece1d2f443708aa542a62100dfcd/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.652850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54ecd2-96f3-4940-a097-5429887b6d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fa0dc41e80462a9ba09e4e44ee509a, user_id=46a5ece1d2f443708aa542a62100dfcd, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:12.656122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54ecd2-96f3-4940-a097-5429887b6d5a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:12.671936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 811/1637] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:12 2022] PUT /identity/v3/projects/70fa0dc41e80462a9ba09e4e44ee509a/users/46a5ece1d2f443708aa542a62100dfcd/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:12.673658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da3e6f-c2b2-4152-a81d-d6f010146ba0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.674040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da3e6f-c2b2-4152-a81d-d6f010146ba0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.674863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da3e6f-c2b2-4152-a81d-d6f010146ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.744757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b3da3e6f-c2b2-4152-a81d-d6f010146ba0 None None] MFA Rules not processed for user `9e5eadb19da7460bb7584d213c5b216b`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:12.747030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b3da3e6f-c2b2-4152-a81d-d6f010146ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:12.788917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 827/1638] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 09:58:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:12.791205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cd6f3-8efa-40ef-b136-1009503632f2 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.791770 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cd6f3-8efa-40ef-b136-1009503632f2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.792325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cd6f3-8efa-40ef-b136-1009503632f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.847164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 812/1639] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:12.848324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da3e6f-c2b2-4152-a81d-d6f010146ba0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:12.881303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-182cd6f3-8efa-40ef-b136-1009503632f2 None None] MFA Rules not processed for user `46a5ece1d2f443708aa542a62100dfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:12.883059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-182cd6f3-8efa-40ef-b136-1009503632f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:12.918473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da3e6f-c2b2-4152-a81d-d6f010146ba0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:12.922963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258efa4e-4a36-4642-aa56-899d78ed89f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:12.924323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258efa4e-4a36-4642-aa56-899d78ed89f0 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:12.924780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258efa4e-4a36-4642-aa56-899d78ed89f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:12.925176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258efa4e-4a36-4642-aa56-899d78ed89f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:12.998822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:13.012522 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 828/1640] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:13.012522 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cd6f3-8efa-40ef-b136-1009503632f2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:13.020650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cd6f3-8efa-40ef-b136-1009503632f2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:13.031839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cea22-674b-4712-9332-b8db7fe03c0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:13.033084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cea22-674b-4712-9332-b8db7fe03c0a service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:13.034331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cea22-674b-4712-9332-b8db7fe03c0a service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:13.035191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cea22-674b-4712-9332-b8db7fe03c0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:13.061329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cea22-674b-4712-9332-b8db7fe03c0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:13.065408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cea22-674b-4712-9332-b8db7fe03c0a service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:13.067296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258efa4e-4a36-4642-aa56-899d78ed89f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:13.070608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258efa4e-4a36-4642-aa56-899d78ed89f0 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:13.097354 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 813/1641] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:13.100709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 829/1642] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:13.101256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389226e-2831-48e8-8ce4-004208bb53a4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:13.102305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389226e-2831-48e8-8ce4-004208bb53a4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:13.102988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389226e-2831-48e8-8ce4-004208bb53a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:13.178391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f389226e-2831-48e8-8ce4-004208bb53a4 None None] MFA Rules not processed for user `d2ed7dd8d03c4c42925e3efca1915a46`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:13.180491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f389226e-2831-48e8-8ce4-004208bb53a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:13.371669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 814/1643] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:13.393243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414b051-cfb4-4e55-ac86-2faa16c5d760 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:13.393707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414b051-cfb4-4e55-ac86-2faa16c5d760 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:13.397718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414b051-cfb4-4e55-ac86-2faa16c5d760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:13.398940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389226e-2831-48e8-8ce4-004208bb53a4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:13.424846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389226e-2831-48e8-8ce4-004208bb53a4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:13.435583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da76de-9c3e-4b1d-a812-701fe8ac1323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:13.437470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da76de-9c3e-4b1d-a812-701fe8ac1323 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:13.438949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da76de-9c3e-4b1d-a812-701fe8ac1323 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:13.440570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da76de-9c3e-4b1d-a812-701fe8ac1323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:13.452111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f414b051-cfb4-4e55-ac86-2faa16c5d760 None None] MFA Rules not processed for user `d2ed7dd8d03c4c42925e3efca1915a46`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:13.453275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f414b051-cfb4-4e55-ac86-2faa16c5d760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:13.462795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da76de-9c3e-4b1d-a812-701fe8ac1323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:13.469104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da76de-9c3e-4b1d-a812-701fe8ac1323 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:13.480989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 830/1644] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:58:13.517103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 815/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:58:13.541925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:14.018702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414b051-cfb4-4e55-ac86-2faa16c5d760 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:14.041300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414b051-cfb4-4e55-ac86-2faa16c5d760 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:14.056574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9641e-211a-415b-8390-e17d8f58c6e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:14.056574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9641e-211a-415b-8390-e17d8f58c6e6 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:14.056865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9641e-211a-415b-8390-e17d8f58c6e6 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:14.057344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9641e-211a-415b-8390-e17d8f58c6e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:14.074951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9641e-211a-415b-8390-e17d8f58c6e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:14.079729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9641e-211a-415b-8390-e17d8f58c6e6 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:14.103148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 831/1646] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:14.164433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da76de-9c3e-4b1d-a812-701fe8ac1323 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:14.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da76de-9c3e-4b1d-a812-701fe8ac1323 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:14.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74a9ab-0d42-4d24-8384-85e15f59c464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:14.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74a9ab-0d42-4d24-8384-85e15f59c464 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:14.419209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74a9ab-0d42-4d24-8384-85e15f59c464 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:14.421118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74a9ab-0d42-4d24-8384-85e15f59c464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:14.421118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74a9ab-0d42-4d24-8384-85e15f59c464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:14.421118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74a9ab-0d42-4d24-8384-85e15f59c464 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:14.421118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 816/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:58:14.656922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8fea-1163-46af-b325-5f3e72fc40e1 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:14.657161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8fea-1163-46af-b325-5f3e72fc40e1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:14.657446 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8fea-1163-46af-b325-5f3e72fc40e1 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:14.659154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 832/1648] 10.222.0.42 () {54 vars in 912 bytes} [Fri Jun 3 09:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:58:14.664278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0a167-2482-46f9-83b2-453975f776e0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:14.664575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0a167-2482-46f9-83b2-453975f776e0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:14.664867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0a167-2482-46f9-83b2-453975f776e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:14.752815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f1e0a167-2482-46f9-83b2-453975f776e0 None None] MFA Rules not processed for user `362797538cbd48dc9c73b09b0548959e`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:14.754136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f1e0a167-2482-46f9-83b2-453975f776e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:14.934478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 817/1649] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 09:58:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:15.026680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8fea-1163-46af-b325-5f3e72fc40e1 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:15.050615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8fea-1163-46af-b325-5f3e72fc40e1 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:15.084190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0b233-58f4-4ae2-be59-a132b9ef6fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:15.086611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0b233-58f4-4ae2-be59-a132b9ef6fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:15.087730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0b233-58f4-4ae2-be59-a132b9ef6fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:15.088392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0b233-58f4-4ae2-be59-a132b9ef6fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:15.111666 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0b233-58f4-4ae2-be59-a132b9ef6fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:15.122690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0b233-58f4-4ae2-be59-a132b9ef6fc1 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:15.140458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 833/1650] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:16.249682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0a167-2482-46f9-83b2-453975f776e0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:16.298471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:16.375761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0a167-2482-46f9-83b2-453975f776e0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:16.384930 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b60eca-2f2c-4e64-b4c3-2a4051dc7063 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:16.385857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b60eca-2f2c-4e64-b4c3-2a4051dc7063 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:16.386429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b60eca-2f2c-4e64-b4c3-2a4051dc7063 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:16.388775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 834/1651] 10.222.0.42 () {54 vars in 960 bytes} [Fri Jun 3 09:58:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:58:16.390387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:16.391426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:16.391712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:16.398805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] PATH_INFO: `/v3/users/1d7675d5d55e43f7940c9ea5b213b559` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:16.398805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d7675d5d55e43f7940c9ea5b213b559)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:16.398805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:16.407358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78442071-bca0-4de8-b015-1821e975bb4f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:16.408161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78442071-bca0-4de8-b015-1821e975bb4f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:16.408689 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78442071-bca0-4de8-b015-1821e975bb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:16.507830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-78442071-bca0-4de8-b015-1821e975bb4f None None] MFA Rules not processed for user `566c0c4bdcaf4c13b060af4bbff5bc04`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:16.507830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-78442071-bca0-4de8-b015-1821e975bb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:16.672639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d7675d5d55e43f7940c9ea5b213b559'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:16.674328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d7675d5d55e43f7940c9ea5b213b559'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:16.750510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d7675d5d55e43f7940c9ea5b213b559'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:16.781466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d7675d5d55e43f7940c9ea5b213b559'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:16.799992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 835/1652] 10.222.0.42 () {58 vars in 1057 bytes} [Fri Jun 3 09:58:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:16.810956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78442071-bca0-4de8-b015-1821e975bb4f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:16.838971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78442071-bca0-4de8-b015-1821e975bb4f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:16.862688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:16.863354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:16.863781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:16.866039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:16.879722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:16.914391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9a2925e4-9b8c-49de-9e82-6a37f604ccaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d7675d5d55e43f7940c9ea5b213b559'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:16.937153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 818/1653] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:16 2022] DELETE /identity/v3/users/1d7675d5d55e43f7940c9ea5b213b559 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:16.941709 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2ef99-1d20-49ed-ab32-ae1f71693a99 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:16.942663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2ef99-1d20-49ed-ab32-ae1f71693a99 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:16.942943 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2ef99-1d20-49ed-ab32-ae1f71693a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:16.953125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:16.964408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:17.004758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-45b2ef99-1d20-49ed-ab32-ae1f71693a99 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:17.007755 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-45b2ef99-1d20-49ed-ab32-ae1f71693a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:17.008917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 836/1654] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:58:17.141118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 819/1655] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:58:17.175322 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:17.176984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:17.203694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadc51a-7f5b-400d-b038-0323526a826b service placement] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:17.328086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334480b7-c3da-40cf-b392-2b697f08b22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:17.329504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334480b7-c3da-40cf-b392-2b697f08b22f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:17.329795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334480b7-c3da-40cf-b392-2b697f08b22f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:17.330164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334480b7-c3da-40cf-b392-2b697f08b22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:17.374155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334480b7-c3da-40cf-b392-2b697f08b22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:17.374155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334480b7-c3da-40cf-b392-2b697f08b22f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:17.394036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 837/1656] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:19.114257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2ef99-1d20-49ed-ab32-ae1f71693a99 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:19.116258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-45b2ef99-1d20-49ed-ab32-ae1f71693a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:19.136322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2ef99-1d20-49ed-ab32-ae1f71693a99 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:19.145314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:19.147055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:19.147706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:19.148212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] PATH_INFO: `/v3/projects/3c94df3549d74809933492001a652f2b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:19.151333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c94df3549d74809933492001a652f2b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:19.153170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: WARNING py.warnings [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:58:19.153170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: warnings.warn(msg) Jun 03 09:58:19.153170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: Jun 03 09:58:19.155177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:19.314970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:19.349605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c94df3549d74809933492001a652f2b', 'request_id': 'req-285cc962-7af1-466e-b7fe-27c1b677c6fb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:19.376328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c94df3549d74809933492001a652f2b', 'request_id': 'req-285cc962-7af1-466e-b7fe-27c1b677c6fb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:19.506426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c94df3549d74809933492001a652f2b', 'request_id': 'req-285cc962-7af1-466e-b7fe-27c1b677c6fb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:19.541171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] The token cache is being invalidate because project 3c94df3549d74809933492001a652f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:19.541463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:19.542323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:19.547472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 820/1657] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:19 2022] DELETE /identity/v3/projects/3c94df3549d74809933492001a652f2b => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:19.559161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334480b7-c3da-40cf-b392-2b697f08b22f service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:19.562215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334480b7-c3da-40cf-b392-2b697f08b22f service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:19.653209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:19.709551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:19.719408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:19.719747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:19.720169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] PATH_INFO: `/v3/users/46a5ece1d2f443708aa542a62100dfcd` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:19.722562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a5ece1d2f443708aa542a62100dfcd)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:19.723615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: WARNING py.warnings [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 09:58:19.723615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: warnings.warn(msg) Jun 03 09:58:19.723615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: Jun 03 09:58:19.732098 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:20.084963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a5ece1d2f443708aa542a62100dfcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:20.085774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a5ece1d2f443708aa542a62100dfcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:20.116443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a5ece1d2f443708aa542a62100dfcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:20.174016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a5ece1d2f443708aa542a62100dfcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:20.283148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d23892e0-37d8-4e5c-b1e7-936d6279716c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a5ece1d2f443708aa542a62100dfcd'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:20.295600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 838/1658] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:19 2022] DELETE /identity/v3/users/46a5ece1d2f443708aa542a62100dfcd => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:20.608963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:20.617743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:20.650737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cc962-7af1-466e-b7fe-27c1b677c6fb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:20.798405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:20.800272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:20.800764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:20.801270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] PATH_INFO: `/v3/users/7031f44ed4c64aa8b8e236cb8e3c1b9e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:20.804232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7031f44ed4c64aa8b8e236cb8e3c1b9e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:20.807128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:21.128992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7031f44ed4c64aa8b8e236cb8e3c1b9e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:21.141788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7031f44ed4c64aa8b8e236cb8e3c1b9e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:21.233778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7031f44ed4c64aa8b8e236cb8e3c1b9e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:21.255480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7031f44ed4c64aa8b8e236cb8e3c1b9e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:21.344101 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7031f44ed4c64aa8b8e236cb8e3c1b9e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:21.351546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 821/1659] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:20 2022] DELETE /identity/v3/users/7031f44ed4c64aa8b8e236cb8e3c1b9e => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:21.355980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b68e7-43fe-40a3-a1f8-a3be7aede4a4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:21.356964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b68e7-43fe-40a3-a1f8-a3be7aede4a4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:21.357418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b68e7-43fe-40a3-a1f8-a3be7aede4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:21.405832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b31b68e7-43fe-40a3-a1f8-a3be7aede4a4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:21.408229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b31b68e7-43fe-40a3-a1f8-a3be7aede4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:21.657275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 839/1660] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:21.692666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:21.694172 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:21.788438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:21.797113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ea52c-64ac-447a-b81a-f968cce53acb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:21.801100 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d5929-99bc-48a6-acc3-a76b74b2aaac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:21.801765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d5929-99bc-48a6-acc3-a76b74b2aaac service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:21.802083 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d5929-99bc-48a6-acc3-a76b74b2aaac service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:21.802364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d5929-99bc-48a6-acc3-a76b74b2aaac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:21.815283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d5929-99bc-48a6-acc3-a76b74b2aaac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:21.817281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d5929-99bc-48a6-acc3-a76b74b2aaac service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:21.832110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 822/1661] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:21.974515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d9936-4f97-49e8-b3fa-43dbe2b9f419 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:21.974948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d9936-4f97-49e8-b3fa-43dbe2b9f419 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:21.975246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d9936-4f97-49e8-b3fa-43dbe2b9f419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:22.016949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-086d9936-4f97-49e8-b3fa-43dbe2b9f419 None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:22.018545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-086d9936-4f97-49e8-b3fa-43dbe2b9f419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:22.287948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 840/1662] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 09:58:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:22.544783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d5929-99bc-48a6-acc3-a76b74b2aaac service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:22.549022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ae4d5929-99bc-48a6-acc3-a76b74b2aaac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:22.598738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d5929-99bc-48a6-acc3-a76b74b2aaac service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:22.618109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:22.622092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:22.627108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:22.627108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] PATH_INFO: `/v3/projects/70fa0dc41e80462a9ba09e4e44ee509a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:22.631683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70fa0dc41e80462a9ba09e4e44ee509a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:22.637983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:22.822630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d9936-4f97-49e8-b3fa-43dbe2b9f419 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:22.824358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-086d9936-4f97-49e8-b3fa-43dbe2b9f419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:22.832972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:22.878002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70fa0dc41e80462a9ba09e4e44ee509a', 'request_id': 'req-8e209045-dcad-4934-96d1-7c166b811190'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:22.893163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70fa0dc41e80462a9ba09e4e44ee509a', 'request_id': 'req-8e209045-dcad-4934-96d1-7c166b811190'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:22.991304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d9936-4f97-49e8-b3fa-43dbe2b9f419 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:23.065337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70fa0dc41e80462a9ba09e4e44ee509a', 'request_id': 'req-8e209045-dcad-4934-96d1-7c166b811190'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:23.093472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] The token cache is being invalidate because project 70fa0dc41e80462a9ba09e4e44ee509a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:23.093804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:23.094506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:23.097971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 823/1663] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:22 2022] DELETE /identity/v3/projects/70fa0dc41e80462a9ba09e4e44ee509a => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:23.098821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:23.106526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:23.126350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43764822-f86a-4cf3-9ab8-f278f0b42dcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:23.127453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43764822-f86a-4cf3-9ab8-f278f0b42dcd service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:23.128240 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43764822-f86a-4cf3-9ab8-f278f0b42dcd service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:23.128713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43764822-f86a-4cf3-9ab8-f278f0b42dcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:23.147830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-43764822-f86a-4cf3-9ab8-f278f0b42dcd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:23.264087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e209045-dcad-4934-96d1-7c166b811190 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:23.277256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:23.278253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:23.278673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:23.278971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] PATH_INFO: `/v3/users/a455c6862ccb4a988fd2ab6de3195cac` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:23.283185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a455c6862ccb4a988fd2ab6de3195cac)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:23.286098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:23.348104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43764822-f86a-4cf3-9ab8-f278f0b42dcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:23.349869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43764822-f86a-4cf3-9ab8-f278f0b42dcd service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:23.366433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 841/1664] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 09:58:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 09:58:23.368315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917c909-997d-4237-b70c-6407f57d5e4e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:23.368714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917c909-997d-4237-b70c-6407f57d5e4e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:23.369088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917c909-997d-4237-b70c-6407f57d5e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:23.408618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a917c909-997d-4237-b70c-6407f57d5e4e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:23.409653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a917c909-997d-4237-b70c-6407f57d5e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:23.431450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 842/1665] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:23.437987 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c5f16-53e7-4cdd-bc05-1ff4f96c42b6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:23.437987 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c5f16-53e7-4cdd-bc05-1ff4f96c42b6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:23.437987 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c5f16-53e7-4cdd-bc05-1ff4f96c42b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:23.517788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6e1c5f16-53e7-4cdd-bc05-1ff4f96c42b6 None None] MFA Rules not processed for user `6113d715500240098217caf641db763a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:23.535936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6e1c5f16-53e7-4cdd-bc05-1ff4f96c42b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:23.559113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a455c6862ccb4a988fd2ab6de3195cac'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:23.559113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a455c6862ccb4a988fd2ab6de3195cac'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:23.605646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a455c6862ccb4a988fd2ab6de3195cac'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:23.634510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a455c6862ccb4a988fd2ab6de3195cac'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:23.638537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:23.649226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:23.714185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f9601b69-f07e-4077-bfb7-26e574beb337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a455c6862ccb4a988fd2ab6de3195cac'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:23.752842 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 824/1666] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:23 2022] DELETE /identity/v3/users/a455c6862ccb4a988fd2ab6de3195cac => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:23.754964 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6a488-d237-4682-bc9e-97cdf3f5107c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:23.755262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6a488-d237-4682-bc9e-97cdf3f5107c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:23.758873 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6a488-d237-4682-bc9e-97cdf3f5107c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:23.784478 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 843/1667] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:23.785871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb773d8e-831f-4dd5-b9ab-31c4c571cc75 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:23.786197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb773d8e-831f-4dd5-b9ab-31c4c571cc75 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:23.789958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6bb6a488-d237-4682-bc9e-97cdf3f5107c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:23.791444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6bb6a488-d237-4682-bc9e-97cdf3f5107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:23.795405 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb773d8e-831f-4dd5-b9ab-31c4c571cc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:23.843236 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-cb773d8e-831f-4dd5-b9ab-31c4c571cc75 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:23.847179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cb773d8e-831f-4dd5-b9ab-31c4c571cc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:23.954895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 825/1668] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:23.956386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5f0d6-129d-4208-a7b2-91d31856f2ab None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:23.956826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5f0d6-129d-4208-a7b2-91d31856f2ab None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:23.958105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5f0d6-129d-4208-a7b2-91d31856f2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:24.034899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 844/1669] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:24.036828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb773d8e-831f-4dd5-b9ab-31c4c571cc75 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:24.043671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cb773d8e-831f-4dd5-b9ab-31c4c571cc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:24.046305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-33a5f0d6-129d-4208-a7b2-91d31856f2ab None None] MFA Rules not processed for user `6113d715500240098217caf641db763a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:24.049018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-33a5f0d6-129d-4208-a7b2-91d31856f2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:24.082581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb773d8e-831f-4dd5-b9ab-31c4c571cc75 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:24.098288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:24.099460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:24.099804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:24.100093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] PATH_INFO: `/v3/projects/b6522d9aaf5a45b2910df87d13a0b30d` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:24.101487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6522d9aaf5a45b2910df87d13a0b30d)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:24.103763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:24.260176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 826/1670] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:24.263086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5f0d6-129d-4208-a7b2-91d31856f2ab None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:24.281840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6522d9aaf5a45b2910df87d13a0b30d', 'request_id': 'req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:24.289281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6522d9aaf5a45b2910df87d13a0b30d', 'request_id': 'req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:24.307495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5f0d6-129d-4208-a7b2-91d31856f2ab None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:24.374219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6522d9aaf5a45b2910df87d13a0b30d', 'request_id': 'req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:24.395521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] The token cache is being invalidate because project b6522d9aaf5a45b2910df87d13a0b30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:24.395891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:24.396948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:24.405646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 845/1671] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:24 2022] DELETE /identity/v3/projects/b6522d9aaf5a45b2910df87d13a0b30d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:24.405646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:24.409078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:24.458489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:24.460256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:24.461259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:24.462652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:24.465163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:24.469499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:24.471451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:24.502096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 827/1672] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:24.503563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:24.508192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:24.635169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7fb3e-f128-48d4-8b35-13b2a0a15ecf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:24.659615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e7333-4bfb-4f7b-bb12-cd4d0c25539e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:24.663345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e7333-4bfb-4f7b-bb12-cd4d0c25539e service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:24.666088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e7333-4bfb-4f7b-bb12-cd4d0c25539e service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:24.670443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e7333-4bfb-4f7b-bb12-cd4d0c25539e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:24.679782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b80e7333-4bfb-4f7b-bb12-cd4d0c25539e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:24.691322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:24.780634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e7333-4bfb-4f7b-bb12-cd4d0c25539e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:24.785681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e7333-4bfb-4f7b-bb12-cd4d0c25539e service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:24.814817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:24.833220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 846/1673] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:24.839590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3ae4f-eee7-4539-ad55-d5e7a2f9c866 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:24.839871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3ae4f-eee7-4539-ad55-d5e7a2f9c866 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:24.840848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3ae4f-eee7-4539-ad55-d5e7a2f9c866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:24.859533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89f8d8-0954-4030-9a4d-5f5577887df4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:24.866153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb4533-2bb2-4800-b6f7-5a393fcc72c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:24.866829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb4533-2bb2-4800-b6f7-5a393fcc72c2 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:24.868158 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb4533-2bb2-4800-b6f7-5a393fcc72c2 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:24.868158 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb4533-2bb2-4800-b6f7-5a393fcc72c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:24.874860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bcdb4533-2bb2-4800-b6f7-5a393fcc72c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:24.935963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-dcc3ae4f-eee7-4539-ad55-d5e7a2f9c866 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:24.940068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dcc3ae4f-eee7-4539-ad55-d5e7a2f9c866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:24.980512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 847/1674] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:24.981381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3ae4f-eee7-4539-ad55-d5e7a2f9c866 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:24.989015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dcc3ae4f-eee7-4539-ad55-d5e7a2f9c866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:25.046846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb4533-2bb2-4800-b6f7-5a393fcc72c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:25.047628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3ae4f-eee7-4539-ad55-d5e7a2f9c866 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:25.048485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb4533-2bb2-4800-b6f7-5a393fcc72c2 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:25.079551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9243128-f5d4-4311-8384-cfe69b5d167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:25.082410 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9243128-f5d4-4311-8384-cfe69b5d167c admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:25.082611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9243128-f5d4-4311-8384-cfe69b5d167c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:25.083177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9243128-f5d4-4311-8384-cfe69b5d167c admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:25.088016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9243128-f5d4-4311-8384-cfe69b5d167c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:25.090322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 828/1675] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:25.091852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2c0b-a009-4fee-bdcb-e368699155b9 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:25.092136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2c0b-a009-4fee-bdcb-e368699155b9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:25.094148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9243128-f5d4-4311-8384-cfe69b5d167c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:25.094605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2c0b-a009-4fee-bdcb-e368699155b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:25.206149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2a8d2c0b-a009-4fee-bdcb-e368699155b9 None None] MFA Rules not processed for user `49efae911ef8458b9288558014217934`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:25.207587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2a8d2c0b-a009-4fee-bdcb-e368699155b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:25.230736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 848/1676] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:24 2022] POST /identity/v3/projects => generated 369 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:25.235027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1ec9-720d-4d1f-8064-1af7754f3343 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:25.235517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1ec9-720d-4d1f-8064-1af7754f3343 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:25.237827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1ec9-720d-4d1f-8064-1af7754f3343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:25.283492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fbec1ec9-720d-4d1f-8064-1af7754f3343 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:25.286873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fbec1ec9-720d-4d1f-8064-1af7754f3343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:25.437027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 829/1677] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:25.438735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d2c0b-a009-4fee-bdcb-e368699155b9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:25.639904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 849/1678] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:25.641981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d2c0b-a009-4fee-bdcb-e368699155b9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:25.641981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df12087-d030-45dd-ad8b-b46e02d24f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:25.641981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df12087-d030-45dd-ad8b-b46e02d24f75 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:25.641981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df12087-d030-45dd-ad8b-b46e02d24f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:25.646079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df12087-d030-45dd-ad8b-b46e02d24f75 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:25.646079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df12087-d030-45dd-ad8b-b46e02d24f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:25.646079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df12087-d030-45dd-ad8b-b46e02d24f75 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:25.647333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec1ec9-720d-4d1f-8064-1af7754f3343 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:25.647333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec1ec9-720d-4d1f-8064-1af7754f3343 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:25.647333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974992ae-cebc-49be-9a21-045ade0d48ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:25.647333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974992ae-cebc-49be-9a21-045ade0d48ef admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:25.649804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974992ae-cebc-49be-9a21-045ade0d48ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:25.649804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974992ae-cebc-49be-9a21-045ade0d48ef admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:25.649804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974992ae-cebc-49be-9a21-045ade0d48ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:25.649804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974992ae-cebc-49be-9a21-045ade0d48ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:25.649804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974992ae-cebc-49be-9a21-045ade0d48ef admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:25.649804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 850/1679] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:58:25.650885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974992ae-cebc-49be-9a21-045ade0d48ef admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:25.669579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974992ae-cebc-49be-9a21-045ade0d48ef admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:25.841214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6533d-4e09-4df0-b425-c3030105ab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:25.841214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6533d-4e09-4df0-b425-c3030105ab1f admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:25.841214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6533d-4e09-4df0-b425-c3030105ab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:25.841214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6533d-4e09-4df0-b425-c3030105ab1f admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:25.842266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6533d-4e09-4df0-b425-c3030105ab1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:25.842266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6533d-4e09-4df0-b425-c3030105ab1f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:25.973970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 830/1680] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:25 2022] POST /identity/v3/users => generated 285 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:25.975289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df12087-d030-45dd-ad8b-b46e02d24f75 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:26.035127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 851/1681] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:25 2022] POST /identity/v3/projects => generated 385 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:26.037676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6533d-4e09-4df0-b425-c3030105ab1f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:26.054352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df12087-d030-45dd-ad8b-b46e02d24f75 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:26.187130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:26.255556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d7283-2793-449c-a6c5-d277b4ea1e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:26.257343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d7283-2793-449c-a6c5-d277b4ea1e96 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:26.257559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d7283-2793-449c-a6c5-d277b4ea1e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:26.257803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d7283-2793-449c-a6c5-d277b4ea1e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:26.259284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d7283-2793-449c-a6c5-d277b4ea1e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:26.260914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d7283-2793-449c-a6c5-d277b4ea1e96 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:26.295897 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6533d-4e09-4df0-b425-c3030105ab1f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:26.309234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 831/1682] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:58:26.313046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc9dc6-79f8-4a54-ba5c-4e39a33df9a0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:26.313403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc9dc6-79f8-4a54-ba5c-4e39a33df9a0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:26.316804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc9dc6-79f8-4a54-ba5c-4e39a33df9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:26.323779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc09b19-86bb-46a9-9b24-b458f05f00ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:26.324483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc09b19-86bb-46a9-9b24-b458f05f00ea admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:26.324768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc09b19-86bb-46a9-9b24-b458f05f00ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:26.325047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc09b19-86bb-46a9-9b24-b458f05f00ea admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:26.325768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc09b19-86bb-46a9-9b24-b458f05f00ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:26.346852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-92bc9dc6-79f8-4a54-ba5c-4e39a33df9a0 None None] MFA Rules not processed for user `362797538cbd48dc9c73b09b0548959e`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:26.349421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-92bc9dc6-79f8-4a54-ba5c-4e39a33df9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:26.356256 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc09b19-86bb-46a9-9b24-b458f05f00ea admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:26.466231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 852/1683] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:26 2022] POST /identity/v3/users => generated 293 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:26.472153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc09b19-86bb-46a9-9b24-b458f05f00ea admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:26.474566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6cc09b19-86bb-46a9-9b24-b458f05f00ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:26.555876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 832/1684] 10.222.0.59 () {58 vars in 1015 bytes} [Fri Jun 3 09:58:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:26.561449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49f92c-6da5-453d-b88b-bd436fbdc725 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:26.561768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49f92c-6da5-453d-b88b-bd436fbdc725 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:26.562064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49f92c-6da5-453d-b88b-bd436fbdc725 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:26.579893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 833/1685] 10.222.0.42 () {54 vars in 912 bytes} [Fri Jun 3 09:58:26 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:58:26.580238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49f92c-6da5-453d-b88b-bd436fbdc725 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:26.585477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:26.625843 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49f92c-6da5-453d-b88b-bd436fbdc725 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:26.632102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c96759-3d14-410a-95c1-8459b7a49163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:26.632792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c96759-3d14-410a-95c1-8459b7a49163 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:26.633079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c96759-3d14-410a-95c1-8459b7a49163 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:26.634145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c96759-3d14-410a-95c1-8459b7a49163 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:26.634956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c96759-3d14-410a-95c1-8459b7a49163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:26.642385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c96759-3d14-410a-95c1-8459b7a49163 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:26.677368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 834/1686] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:26.677368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c96759-3d14-410a-95c1-8459b7a49163 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:26.704097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c96759-3d14-410a-95c1-8459b7a49163 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:26.752886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2d5ae-4739-40ae-9719-8d7ecb9e53af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:26.753675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2d5ae-4739-40ae-9719-8d7ecb9e53af admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:26.753976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2d5ae-4739-40ae-9719-8d7ecb9e53af admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:26.754255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2d5ae-4739-40ae-9719-8d7ecb9e53af admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:26.754990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2d5ae-4739-40ae-9719-8d7ecb9e53af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:26.756612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2d5ae-4739-40ae-9719-8d7ecb9e53af admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:26.792843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc09b19-86bb-46a9-9b24-b458f05f00ea admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:26.807579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 835/1687] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:26.816690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb47eb-767c-499e-87bc-0a01371bf20b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:26.817137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb47eb-767c-499e-87bc-0a01371bf20b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:26.817270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb47eb-767c-499e-87bc-0a01371bf20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:26.827849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02357370-e608-43a0-a54e-4d952b3cda1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:26.828547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02357370-e608-43a0-a54e-4d952b3cda1e service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:26.828827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02357370-e608-43a0-a54e-4d952b3cda1e service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:26.829115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02357370-e608-43a0-a54e-4d952b3cda1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:26.895883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d8eb47eb-767c-499e-87bc-0a01371bf20b None None] MFA Rules not processed for user `362797538cbd48dc9c73b09b0548959e`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:26.898479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d8eb47eb-767c-499e-87bc-0a01371bf20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:26.924960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 836/1688] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 09:58:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:26.926967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb47eb-767c-499e-87bc-0a01371bf20b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:26.929575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d8eb47eb-767c-499e-87bc-0a01371bf20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:26.962227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb47eb-767c-499e-87bc-0a01371bf20b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:26.976240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58102-191b-40bd-89ef-8c01153170f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:26.976240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58102-191b-40bd-89ef-8c01153170f1 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:26.976240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58102-191b-40bd-89ef-8c01153170f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:26.976240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58102-191b-40bd-89ef-8c01153170f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:26.991133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c58102-191b-40bd-89ef-8c01153170f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:26.992946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c58102-191b-40bd-89ef-8c01153170f1 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:27.005022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 837/1689] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:27.007327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58102-191b-40bd-89ef-8c01153170f1 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:27.033839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58102-191b-40bd-89ef-8c01153170f1 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:27.035110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02357370-e608-43a0-a54e-4d952b3cda1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:27.037105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02357370-e608-43a0-a54e-4d952b3cda1e service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:27.048760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a8a60-8edc-421e-b950-42a866f51004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:27.051712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a8a60-8edc-421e-b950-42a866f51004 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:27.053629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a8a60-8edc-421e-b950-42a866f51004 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:27.056298 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a8a60-8edc-421e-b950-42a866f51004 admin admin] PATH_INFO: `/v3/projects/8c58d6b0730c4c4cb619a7774d416b3d/users/b9fae63789d341c3996f8066dca8dfe9/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:27.088865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a8a60-8edc-421e-b950-42a866f51004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c58d6b0730c4c4cb619a7774d416b3d, user_id=b9fae63789d341c3996f8066dca8dfe9, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:27.092537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a8a60-8edc-421e-b950-42a866f51004 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:27.104161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 853/1690] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:27.105376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02357370-e608-43a0-a54e-4d952b3cda1e service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:27.144065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02357370-e608-43a0-a54e-4d952b3cda1e service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:27.167929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fb616-6d9f-4e47-82b8-1b13637cbbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:27.168806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fb616-6d9f-4e47-82b8-1b13637cbbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:27.169409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fb616-6d9f-4e47-82b8-1b13637cbbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:27.170263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fb616-6d9f-4e47-82b8-1b13637cbbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:27.171213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fb616-6d9f-4e47-82b8-1b13637cbbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:27.175875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fb616-6d9f-4e47-82b8-1b13637cbbf2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:27.201939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 854/1691] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:27.226681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fb616-6d9f-4e47-82b8-1b13637cbbf2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:27.226681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fc9fb616-6d9f-4e47-82b8-1b13637cbbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:27.231285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 838/1692] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:27 2022] PUT /identity/v3/projects/8c58d6b0730c4c4cb619a7774d416b3d/users/b9fae63789d341c3996f8066dca8dfe9/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:27.234438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a8a60-8edc-421e-b950-42a866f51004 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:27.266011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fb616-6d9f-4e47-82b8-1b13637cbbf2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:27.402570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a8a60-8edc-421e-b950-42a866f51004 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:27.428368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214f926-71b9-4306-aa6b-ff64caaed535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:27.428368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214f926-71b9-4306-aa6b-ff64caaed535 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:27.428368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214f926-71b9-4306-aa6b-ff64caaed535 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:27.428368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214f926-71b9-4306-aa6b-ff64caaed535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:27.450732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214f926-71b9-4306-aa6b-ff64caaed535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:27.452315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214f926-71b9-4306-aa6b-ff64caaed535 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:27.485096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 839/1693] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:27.488111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b00f3a-a167-4762-8dda-aecf8437eb08 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:27.489587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b00f3a-a167-4762-8dda-aecf8437eb08 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:27.490795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b00f3a-a167-4762-8dda-aecf8437eb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:27.538623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:27.541043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:27.542566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:27.542566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] PATH_INFO: `/v3/projects/2887f7ffd32547458585734b104e66b5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:27.545366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2887f7ffd32547458585734b104e66b5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:27.548239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:27.568518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-05b00f3a-a167-4762-8dda-aecf8437eb08 None None] MFA Rules not processed for user `5a304d58eff84a87953dc7af6a0afe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:27.570073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-05b00f3a-a167-4762-8dda-aecf8437eb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:27.602291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:27.748986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 840/1694] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:27.748986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b00f3a-a167-4762-8dda-aecf8437eb08 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:27.771937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b00f3a-a167-4762-8dda-aecf8437eb08 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:27.783877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2887f7ffd32547458585734b104e66b5', 'request_id': 'req-ae974695-2192-440d-aebf-3e329fa7a9c7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:27.792443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:27.811183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2887f7ffd32547458585734b104e66b5', 'request_id': 'req-ae974695-2192-440d-aebf-3e329fa7a9c7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:27.916307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2887f7ffd32547458585734b104e66b5', 'request_id': 'req-ae974695-2192-440d-aebf-3e329fa7a9c7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:27.930700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865445c7-469d-4b29-8f95-0af7847521b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:27.933753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865445c7-469d-4b29-8f95-0af7847521b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:27.933753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865445c7-469d-4b29-8f95-0af7847521b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:27.933753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865445c7-469d-4b29-8f95-0af7847521b6 admin admin] PATH_INFO: `/v3/projects/901c8919b73c4680877d340ca513ed68/users/6cb057ec20ac454d96170dafe0c68bf0/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:27.948486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] The token cache is being invalidate because project 2887f7ffd32547458585734b104e66b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:27.949305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:27.956710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ae974695-2192-440d-aebf-3e329fa7a9c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:27.976142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 855/1695] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:27 2022] DELETE /identity/v3/projects/2887f7ffd32547458585734b104e66b5 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:27.978816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a764a30-cf3e-4412-9cbd-12e212292bcf None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:27.980543 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a764a30-cf3e-4412-9cbd-12e212292bcf None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:27.980840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a764a30-cf3e-4412-9cbd-12e212292bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:27.987072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865445c7-469d-4b29-8f95-0af7847521b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901c8919b73c4680877d340ca513ed68, user_id=6cb057ec20ac454d96170dafe0c68bf0, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:27.995911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865445c7-469d-4b29-8f95-0af7847521b6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:28.077127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5a764a30-cf3e-4412-9cbd-12e212292bcf None None] MFA Rules not processed for user `b9fae63789d341c3996f8066dca8dfe9`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:28.084859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5a764a30-cf3e-4412-9cbd-12e212292bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:28.123154 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 841/1696] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:27 2022] PUT /identity/v3/projects/901c8919b73c4680877d340ca513ed68/users/6cb057ec20ac454d96170dafe0c68bf0/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:28.130081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2444e-1f35-4dcb-a012-2629b9ee077b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:28.130081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2444e-1f35-4dcb-a012-2629b9ee077b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:28.130081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2444e-1f35-4dcb-a012-2629b9ee077b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:28.229414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-74b2444e-1f35-4dcb-a012-2629b9ee077b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:28.231939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-74b2444e-1f35-4dcb-a012-2629b9ee077b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:28.317036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 856/1697] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:28.322143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbe77e-0803-4b57-a4fa-192cf49bce54 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:28.323793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbe77e-0803-4b57-a4fa-192cf49bce54 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:28.324243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbe77e-0803-4b57-a4fa-192cf49bce54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:28.382110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 842/1698] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:28.383173 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2444e-1f35-4dcb-a012-2629b9ee077b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:28.392802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-74b2444e-1f35-4dcb-a012-2629b9ee077b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:28.400981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-57bbe77e-0803-4b57-a4fa-192cf49bce54 None None] MFA Rules not processed for user `6cb057ec20ac454d96170dafe0c68bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:28.402892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-57bbe77e-0803-4b57-a4fa-192cf49bce54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:28.430347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2444e-1f35-4dcb-a012-2629b9ee077b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:28.449204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2578b-671a-4e2a-ac02-c24f668b3de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:28.454316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2578b-671a-4e2a-ac02-c24f668b3de2 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:28.454316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2578b-671a-4e2a-ac02-c24f668b3de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:28.454672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2578b-671a-4e2a-ac02-c24f668b3de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:28.455855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2578b-671a-4e2a-ac02-c24f668b3de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:28.457383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2578b-671a-4e2a-ac02-c24f668b3de2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:28.572191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 843/1699] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:28 2022] POST /identity/v3/projects => generated 367 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:28.575638 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566689c-1c93-44ee-861c-47c3468401a5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:28.578823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566689c-1c93-44ee-861c-47c3468401a5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:28.579198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566689c-1c93-44ee-861c-47c3468401a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:28.610174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 857/1700] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:28.610939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbe77e-0803-4b57-a4fa-192cf49bce54 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:28.623524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-57bbe77e-0803-4b57-a4fa-192cf49bce54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:28.639910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1566689c-1c93-44ee-861c-47c3468401a5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:28.643127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1566689c-1c93-44ee-861c-47c3468401a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:28.700115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:28.727313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:28.774315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbe77e-0803-4b57-a4fa-192cf49bce54 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:29.004507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 844/1701] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:29.005856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566689c-1c93-44ee-861c-47c3468401a5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:29.045026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566689c-1c93-44ee-861c-47c3468401a5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:29.051071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f864df-e9c2-4331-baf2-36597ea243b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:29.052430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f864df-e9c2-4331-baf2-36597ea243b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:29.061368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f864df-e9c2-4331-baf2-36597ea243b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:29.061654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f864df-e9c2-4331-baf2-36597ea243b2 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:29.062575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f864df-e9c2-4331-baf2-36597ea243b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:29.064735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c8219-8aa1-4244-8e29-8a4c55dab8f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:29.065072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f864df-e9c2-4331-baf2-36597ea243b2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:29.070195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c8219-8aa1-4244-8e29-8a4c55dab8f8 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:29.071497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c8219-8aa1-4244-8e29-8a4c55dab8f8 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:29.072637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c8219-8aa1-4244-8e29-8a4c55dab8f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:29.076000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-998c8219-8aa1-4244-8e29-8a4c55dab8f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:29.263641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 845/1702] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:29 2022] POST /identity/v3/users => generated 290 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:29.266559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f864df-e9c2-4331-baf2-36597ea243b2 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:29.269813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e6f864df-e9c2-4331-baf2-36597ea243b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:29.281668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c8219-8aa1-4244-8e29-8a4c55dab8f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:29.284251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c8219-8aa1-4244-8e29-8a4c55dab8f8 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:29.299063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f864df-e9c2-4331-baf2-36597ea243b2 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:29.317971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 858/1703] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:29.321159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c8219-8aa1-4244-8e29-8a4c55dab8f8 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:29.337044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f16982-d0a1-4433-a26e-27391ba441cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:29.339622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16982-d0a1-4433-a26e-27391ba441cb admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:29.352462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16982-d0a1-4433-a26e-27391ba441cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:29.356965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16982-d0a1-4433-a26e-27391ba441cb admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:29.357436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f16982-d0a1-4433-a26e-27391ba441cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:29.362779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f16982-d0a1-4433-a26e-27391ba441cb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:29.367323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c8219-8aa1-4244-8e29-8a4c55dab8f8 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:29.384648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d5e3e-8d66-4008-914c-dfd647b46b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:29.384975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d5e3e-8d66-4008-914c-dfd647b46b0a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:29.385581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d5e3e-8d66-4008-914c-dfd647b46b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:29.388086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d5e3e-8d66-4008-914c-dfd647b46b0a admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:29.389622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d5e3e-8d66-4008-914c-dfd647b46b0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:29.400810 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d5e3e-8d66-4008-914c-dfd647b46b0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:29.403813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d5e3e-8d66-4008-914c-dfd647b46b0a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:29.439133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 859/1704] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:29.441221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9bc45-7ea6-4f30-be5f-20acdd428fe8 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:29.457391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9bc45-7ea6-4f30-be5f-20acdd428fe8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:29.475193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c9bc45-7ea6-4f30-be5f-20acdd428fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:29.543017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d9c9bc45-7ea6-4f30-be5f-20acdd428fe8 None None] MFA Rules not processed for user `6113d715500240098217caf641db763a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:29.551645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d9c9bc45-7ea6-4f30-be5f-20acdd428fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:29.588587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 846/1705] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:29 2022] POST /identity/v3/projects => generated 387 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:29.588587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f16982-d0a1-4433-a26e-27391ba441cb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:29.688635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 860/1706] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:29.702825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9bc45-7ea6-4f30-be5f-20acdd428fe8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:29.745628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c9bc45-7ea6-4f30-be5f-20acdd428fe8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:29.831310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f16982-d0a1-4433-a26e-27391ba441cb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:29.856355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608d146-c387-4abf-8d2e-c91d68866ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:29.857118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608d146-c387-4abf-8d2e-c91d68866ccd admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:29.858233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608d146-c387-4abf-8d2e-c91d68866ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:29.858587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608d146-c387-4abf-8d2e-c91d68866ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:29.861120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608d146-c387-4abf-8d2e-c91d68866ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:29.868024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608d146-c387-4abf-8d2e-c91d68866ccd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:29.900958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 847/1707] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:29.902838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608d146-c387-4abf-8d2e-c91d68866ccd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:29.961169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608d146-c387-4abf-8d2e-c91d68866ccd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:29.973559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd5608-0fb8-4c35-8f5a-328ff320c497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:29.974907 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd5608-0fb8-4c35-8f5a-328ff320c497 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:29.976143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd5608-0fb8-4c35-8f5a-328ff320c497 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:29.986611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd5608-0fb8-4c35-8f5a-328ff320c497 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:29.987630 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd5608-0fb8-4c35-8f5a-328ff320c497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:29.995234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd5608-0fb8-4c35-8f5a-328ff320c497 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:29.998062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2e195-7ce6-4637-99b9-9e704408af45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:29.998752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2e195-7ce6-4637-99b9-9e704408af45 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:30.006909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2e195-7ce6-4637-99b9-9e704408af45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:30.007297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2e195-7ce6-4637-99b9-9e704408af45 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:30.008054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2e195-7ce6-4637-99b9-9e704408af45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:30.025084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2e195-7ce6-4637-99b9-9e704408af45 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:30.129337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 848/1708] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:29 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:30.131035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8320-8e91-4975-a7ca-47f99e3e8c95 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:30.131527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8320-8e91-4975-a7ca-47f99e3e8c95 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:30.134262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8320-8e91-4975-a7ca-47f99e3e8c95 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:30.136017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 849/1709] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 09:58:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:58:30.144447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8320-8e91-4975-a7ca-47f99e3e8c95 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:30.193933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8320-8e91-4975-a7ca-47f99e3e8c95 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:30.200848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 861/1710] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:29 2022] POST /identity/v3/projects => generated 375 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:30.203073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2e195-7ce6-4637-99b9-9e704408af45 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:30.331677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:30.355732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279d9c6-ba2c-4ae1-b308-3a6b0194979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:30.356937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279d9c6-ba2c-4ae1-b308-3a6b0194979b admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:30.357226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279d9c6-ba2c-4ae1-b308-3a6b0194979b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:30.357506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279d9c6-ba2c-4ae1-b308-3a6b0194979b admin admin] PATH_INFO: `/v3/projects/3ce71271b10842a3aeaa7f2264d20828/users/1a7c8c24ef1f43889aa5d7f3bc556ae0/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:30.450947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279d9c6-ba2c-4ae1-b308-3a6b0194979b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce71271b10842a3aeaa7f2264d20828, user_id=1a7c8c24ef1f43889aa5d7f3bc556ae0, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:30.464032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279d9c6-ba2c-4ae1-b308-3a6b0194979b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:30.484174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2e195-7ce6-4637-99b9-9e704408af45 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:30.491092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5b775-06a8-4185-895d-3b5758ac467d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:30.492500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5b775-06a8-4185-895d-3b5758ac467d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:30.492935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5b775-06a8-4185-895d-3b5758ac467d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:30.493348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5b775-06a8-4185-895d-3b5758ac467d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:30.494239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5b775-06a8-4185-895d-3b5758ac467d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:30.496544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5b775-06a8-4185-895d-3b5758ac467d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:30.522355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:30.541810 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 862/1711] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:30.544049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5952e2-c902-4eaa-aa3b-63f180298884 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:30.544749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5952e2-c902-4eaa-aa3b-63f180298884 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:30.545133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5952e2-c902-4eaa-aa3b-63f180298884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:30.587064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1e5952e2-c902-4eaa-aa3b-63f180298884 None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:30.588621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1e5952e2-c902-4eaa-aa3b-63f180298884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:30.607707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 850/1712] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:30 2022] PUT /identity/v3/projects/3ce71271b10842a3aeaa7f2264d20828/users/1a7c8c24ef1f43889aa5d7f3bc556ae0/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:30.608544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279d9c6-ba2c-4ae1-b308-3a6b0194979b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:30.807068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279d9c6-ba2c-4ae1-b308-3a6b0194979b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:30.824718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9260ad-8d62-4393-80a6-2025b5bbedcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:30.826110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 863/1713] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 09:58:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:30.828279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9260ad-8d62-4393-80a6-2025b5bbedcf admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:30.828574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9260ad-8d62-4393-80a6-2025b5bbedcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:30.828912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9260ad-8d62-4393-80a6-2025b5bbedcf admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:30.829272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5952e2-c902-4eaa-aa3b-63f180298884 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:30.832395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9260ad-8d62-4393-80a6-2025b5bbedcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:30.840121 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9260ad-8d62-4393-80a6-2025b5bbedcf admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:30.864631 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5952e2-c902-4eaa-aa3b-63f180298884 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:30.871075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283b7a7-3308-4663-bb51-aa2144471fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:30.871903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283b7a7-3308-4663-bb51-aa2144471fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:30.872345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283b7a7-3308-4663-bb51-aa2144471fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:30.872755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283b7a7-3308-4663-bb51-aa2144471fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:30.873709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283b7a7-3308-4663-bb51-aa2144471fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:30.875606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283b7a7-3308-4663-bb51-aa2144471fe5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:30.897688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 864/1714] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:30.897688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283b7a7-3308-4663-bb51-aa2144471fe5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:30.944945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283b7a7-3308-4663-bb51-aa2144471fe5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:30.955502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6b177-2fb7-41fa-8d96-85270012a7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:30.957161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6b177-2fb7-41fa-8d96-85270012a7fd admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:30.957611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6b177-2fb7-41fa-8d96-85270012a7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:30.958164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6b177-2fb7-41fa-8d96-85270012a7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:30.959209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6b177-2fb7-41fa-8d96-85270012a7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:30.961764 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6b177-2fb7-41fa-8d96-85270012a7fd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:30.997926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 865/1715] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:30.998822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6b177-2fb7-41fa-8d96-85270012a7fd admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:31.013469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 851/1716] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:30 2022] POST /identity/v3/users => generated 288 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:31.015544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9260ad-8d62-4393-80a6-2025b5bbedcf admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:31.050277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6b177-2fb7-41fa-8d96-85270012a7fd admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:31.050451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9260ad-8d62-4393-80a6-2025b5bbedcf admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:31.067277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68067e-8ec1-4ded-be0f-9a52596cf944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:31.069058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68067e-8ec1-4ded-be0f-9a52596cf944 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:31.070505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68067e-8ec1-4ded-be0f-9a52596cf944 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:31.071714 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68067e-8ec1-4ded-be0f-9a52596cf944 admin admin] PATH_INFO: `/v3/projects/3ce71271b10842a3aeaa7f2264d20828/users/1a7c8c24ef1f43889aa5d7f3bc556ae0/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:31.076271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68067e-8ec1-4ded-be0f-9a52596cf944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce71271b10842a3aeaa7f2264d20828, user_id=1a7c8c24ef1f43889aa5d7f3bc556ae0, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:31.078505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405f96d-f1c5-492c-8ae6-a35ad2ab1ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:31.084564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405f96d-f1c5-492c-8ae6-a35ad2ab1ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:31.085602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405f96d-f1c5-492c-8ae6-a35ad2ab1ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:31.086378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405f96d-f1c5-492c-8ae6-a35ad2ab1ca3 admin admin] PATH_INFO: `/v3/projects/0414b5a62af84263956b6d13ab6ff111/users/6de9d4200d6a46eeb85e58a154b83149/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:31.088317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da68067e-8ec1-4ded-be0f-9a52596cf944 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:31.119186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 852/1717] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:31 2022] PUT /identity/v3/projects/3ce71271b10842a3aeaa7f2264d20828/users/1a7c8c24ef1f43889aa5d7f3bc556ae0/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:31.120243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68067e-8ec1-4ded-be0f-9a52596cf944 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:31.191751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405f96d-f1c5-492c-8ae6-a35ad2ab1ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0414b5a62af84263956b6d13ab6ff111, user_id=6de9d4200d6a46eeb85e58a154b83149, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:31.213851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405f96d-f1c5-492c-8ae6-a35ad2ab1ca3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:31.325272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 866/1718] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:30 2022] PUT /identity/v3/projects/0414b5a62af84263956b6d13ab6ff111/users/6de9d4200d6a46eeb85e58a154b83149/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:31.336979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0521892-33e1-4658-9711-9aa6f1b15715 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:31.337272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0521892-33e1-4658-9711-9aa6f1b15715 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:31.339964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0521892-33e1-4658-9711-9aa6f1b15715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:31.347191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68067e-8ec1-4ded-be0f-9a52596cf944 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:31.400398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:31.465239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d0521892-33e1-4658-9711-9aa6f1b15715 None None] MFA Rules not processed for user `1a7c8c24ef1f43889aa5d7f3bc556ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:31.467917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d0521892-33e1-4658-9711-9aa6f1b15715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:31.514393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ee295-4ffe-4178-9124-372a7e9fedfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:31.517709 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ee295-4ffe-4178-9124-372a7e9fedfa admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:31.518139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ee295-4ffe-4178-9124-372a7e9fedfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:31.518497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ee295-4ffe-4178-9124-372a7e9fedfa admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:31.519303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ee295-4ffe-4178-9124-372a7e9fedfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:31.524236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ee295-4ffe-4178-9124-372a7e9fedfa admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:31.560436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 853/1719] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:31.561719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7728c-e176-4c89-b845-a9adaf119e81 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:31.565563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7728c-e176-4c89-b845-a9adaf119e81 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:31.565926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7728c-e176-4c89-b845-a9adaf119e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:31.658093 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5bc7728c-e176-4c89-b845-a9adaf119e81 None None] MFA Rules not processed for user `6de9d4200d6a46eeb85e58a154b83149`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:31.664926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5bc7728c-e176-4c89-b845-a9adaf119e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:31.681648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 867/1720] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:31.685583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0521892-33e1-4658-9711-9aa6f1b15715 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:31.728931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0521892-33e1-4658-9711-9aa6f1b15715 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:31.748193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafcb8f-83e9-4262-977e-e0147fddbea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:31.748863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafcb8f-83e9-4262-977e-e0147fddbea3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:31.749144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafcb8f-83e9-4262-977e-e0147fddbea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:31.749420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafcb8f-83e9-4262-977e-e0147fddbea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:31.761785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafcb8f-83e9-4262-977e-e0147fddbea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:31.776754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafcb8f-83e9-4262-977e-e0147fddbea3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:31.819570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 868/1721] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:31.820572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafcb8f-83e9-4262-977e-e0147fddbea3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:31.835006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 854/1722] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:31.836192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7728c-e176-4c89-b845-a9adaf119e81 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:31.840556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5bc7728c-e176-4c89-b845-a9adaf119e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:32.018221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7728c-e176-4c89-b845-a9adaf119e81 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:32.034449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404eaa8-3104-4833-95ae-bfac2cc02937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:32.039712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404eaa8-3104-4833-95ae-bfac2cc02937 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:32.042526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404eaa8-3104-4833-95ae-bfac2cc02937 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:32.043351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404eaa8-3104-4833-95ae-bfac2cc02937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:32.071883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404eaa8-3104-4833-95ae-bfac2cc02937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:32.077831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404eaa8-3104-4833-95ae-bfac2cc02937 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:32.107392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 855/1723] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:32.108528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404eaa8-3104-4833-95ae-bfac2cc02937 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:32.154645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:32.172965 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafcb8f-83e9-4262-977e-e0147fddbea3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:32.178512 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404eaa8-3104-4833-95ae-bfac2cc02937 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:32.203545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f788ed-7e18-44d8-b064-ca8f5e17632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:32.203545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f788ed-7e18-44d8-b064-ca8f5e17632d admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:32.205784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f788ed-7e18-44d8-b064-ca8f5e17632d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:32.206620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f788ed-7e18-44d8-b064-ca8f5e17632d admin admin] PATH_INFO: `/v3/projects/710f8018e9e94e8db41725502dce8978/users/4dadf566d4b14f4a84920dd441831022/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:32.245804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0846d-6315-444b-b943-b9674ddae9d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:32.251224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0846d-6315-444b-b943-b9674ddae9d6 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:32.251979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0846d-6315-444b-b943-b9674ddae9d6 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:32.254830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0846d-6315-444b-b943-b9674ddae9d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:32.275469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f788ed-7e18-44d8-b064-ca8f5e17632d admin admin] RBAC: Authorizing `identity:create_grant(project_id=710f8018e9e94e8db41725502dce8978, user_id=4dadf566d4b14f4a84920dd441831022, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:32.278231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0846d-6315-444b-b943-b9674ddae9d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:32.280532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0846d-6315-444b-b943-b9674ddae9d6 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:32.284573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f788ed-7e18-44d8-b064-ca8f5e17632d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:32.407295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 869/1724] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:32.408444 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0846d-6315-444b-b943-b9674ddae9d6 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:32.482610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 856/1725] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:32 2022] PUT /identity/v3/projects/710f8018e9e94e8db41725502dce8978/users/4dadf566d4b14f4a84920dd441831022/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:32.482610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f788ed-7e18-44d8-b064-ca8f5e17632d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:32.485796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0846d-6315-444b-b943-b9674ddae9d6 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:32.821343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb1970-459e-454e-8297-dab22e6cfbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:32.822378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb1970-459e-454e-8297-dab22e6cfbfd admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:32.822819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb1970-459e-454e-8297-dab22e6cfbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:32.825380 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb1970-459e-454e-8297-dab22e6cfbfd admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:32.826765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb1970-459e-454e-8297-dab22e6cfbfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:32.828874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb1970-459e-454e-8297-dab22e6cfbfd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:32.829362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f788ed-7e18-44d8-b064-ca8f5e17632d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:32.851089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4003-1ccf-4b25-ba79-a40fa4dce65b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:32.854968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4003-1ccf-4b25-ba79-a40fa4dce65b service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:32.854968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4003-1ccf-4b25-ba79-a40fa4dce65b service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:32.854968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4003-1ccf-4b25-ba79-a40fa4dce65b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:33.033348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 870/1726] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:32 2022] POST /identity/v3/projects => generated 385 bytes in 625 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:33.036218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da33d1-3086-4561-b192-759f951c241c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:33.037273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da33d1-3086-4561-b192-759f951c241c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:33.040094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da33d1-3086-4561-b192-759f951c241c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:33.054511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4003-1ccf-4b25-ba79-a40fa4dce65b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:33.078626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4003-1ccf-4b25-ba79-a40fa4dce65b service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:33.140273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 857/1727] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:33.141765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4003-1ccf-4b25-ba79-a40fa4dce65b service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:33.194717 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4003-1ccf-4b25-ba79-a40fa4dce65b service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:33.207116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-92da33d1-3086-4561-b192-759f951c241c None None] MFA Rules not processed for user `4dadf566d4b14f4a84920dd441831022`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:33.210115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-92da33d1-3086-4561-b192-759f951c241c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:33.375192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:33.457525 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 871/1728] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:33.471697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da33d1-3086-4561-b192-759f951c241c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:33.473225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-92da33d1-3086-4561-b192-759f951c241c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:33.528636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e215f6-c206-417b-8425-7b557e796739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:33.530543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e215f6-c206-417b-8425-7b557e796739 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:33.530849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e215f6-c206-417b-8425-7b557e796739 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:33.533196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e215f6-c206-417b-8425-7b557e796739 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:33.545847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e215f6-c206-417b-8425-7b557e796739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:33.552398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e215f6-c206-417b-8425-7b557e796739 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:33.631386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da33d1-3086-4561-b192-759f951c241c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:33.648003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe60b2-2eb5-4c5c-a7f4-9f77c20d9d28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:33.650143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe60b2-2eb5-4c5c-a7f4-9f77c20d9d28 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:33.651278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe60b2-2eb5-4c5c-a7f4-9f77c20d9d28 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:33.653678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe60b2-2eb5-4c5c-a7f4-9f77c20d9d28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:33.677779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe60b2-2eb5-4c5c-a7f4-9f77c20d9d28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:33.679861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe60b2-2eb5-4c5c-a7f4-9f77c20d9d28 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:33.688250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 858/1729] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:33 2022] POST /identity/v3/users => generated 299 bytes in 547 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:33.692210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e215f6-c206-417b-8425-7b557e796739 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:33.720267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 872/1730] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:58:33.730484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e215f6-c206-417b-8425-7b557e796739 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:33.756387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b76fe-551b-407f-8e3d-0c531177290e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:33.756387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b76fe-551b-407f-8e3d-0c531177290e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:33.758330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b76fe-551b-407f-8e3d-0c531177290e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:33.759238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b76fe-551b-407f-8e3d-0c531177290e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:33.759887 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b76fe-551b-407f-8e3d-0c531177290e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:33.768535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b76fe-551b-407f-8e3d-0c531177290e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:33.820927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 859/1731] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:33.825111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe60b2-2eb5-4c5c-a7f4-9f77c20d9d28 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:33.856558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe60b2-2eb5-4c5c-a7f4-9f77c20d9d28 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:33.867208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:33.878359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e9530-f203-44f3-8cc0-659d558a96d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:33.880315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e9530-f203-44f3-8cc0-659d558a96d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:33.882242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e9530-f203-44f3-8cc0-659d558a96d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:33.882548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e9530-f203-44f3-8cc0-659d558a96d1 admin admin] PATH_INFO: `/v3/projects/a675122b4da64becb2f0bfc401f2f8df/users/0ecb315931994594a9d61528dc374d95/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:33.923958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62f15c-7c3e-4633-bc13-c9e73987f026 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:33.924294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62f15c-7c3e-4633-bc13-c9e73987f026 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:33.924645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62f15c-7c3e-4633-bc13-c9e73987f026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:33.943309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e9530-f203-44f3-8cc0-659d558a96d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a675122b4da64becb2f0bfc401f2f8df, user_id=0ecb315931994594a9d61528dc374d95, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:33.946032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e9530-f203-44f3-8cc0-659d558a96d1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:34.003908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bd62f15c-7c3e-4633-bc13-c9e73987f026 None None] MFA Rules not processed for user `49efae911ef8458b9288558014217934`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:34.012739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bd62f15c-7c3e-4633-bc13-c9e73987f026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:34.108959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 873/1732] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:33 2022] PUT /identity/v3/projects/a675122b4da64becb2f0bfc401f2f8df/users/0ecb315931994594a9d61528dc374d95/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:34.115570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e9530-f203-44f3-8cc0-659d558a96d1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:34.270155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e9530-f203-44f3-8cc0-659d558a96d1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:34.293060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352f7ef-4d2a-4171-ba25-096663e7874c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:34.294603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352f7ef-4d2a-4171-ba25-096663e7874c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:34.297493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352f7ef-4d2a-4171-ba25-096663e7874c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:34.297493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352f7ef-4d2a-4171-ba25-096663e7874c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:34.297493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352f7ef-4d2a-4171-ba25-096663e7874c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:34.301547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352f7ef-4d2a-4171-ba25-096663e7874c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:34.330303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 874/1733] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:58:34.336506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352f7ef-4d2a-4171-ba25-096663e7874c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:34.352720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 860/1734] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:58:34.356361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352f7ef-4d2a-4171-ba25-096663e7874c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:34.370605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e820783-343b-425a-ba48-dedc9420907c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:34.383863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e820783-343b-425a-ba48-dedc9420907c admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:34.383863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e820783-343b-425a-ba48-dedc9420907c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:34.383863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e820783-343b-425a-ba48-dedc9420907c admin admin] PATH_INFO: `/v3/projects/a675122b4da64becb2f0bfc401f2f8df/users/0ecb315931994594a9d61528dc374d95/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:34.389070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e820783-343b-425a-ba48-dedc9420907c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a675122b4da64becb2f0bfc401f2f8df, user_id=0ecb315931994594a9d61528dc374d95, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:34.391921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62f15c-7c3e-4633-bc13-c9e73987f026 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:34.399363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e820783-343b-425a-ba48-dedc9420907c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:34.421855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62f15c-7c3e-4633-bc13-c9e73987f026 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:34.434754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 875/1735] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:34 2022] PUT /identity/v3/projects/a675122b4da64becb2f0bfc401f2f8df/users/0ecb315931994594a9d61528dc374d95/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:58:34.449310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb493c8-641f-4845-974f-68e688637130 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:34.450009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb493c8-641f-4845-974f-68e688637130 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:34.451157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb493c8-641f-4845-974f-68e688637130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:34.529193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1bb493c8-641f-4845-974f-68e688637130 None None] MFA Rules not processed for user `0ecb315931994594a9d61528dc374d95`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:34.530849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1bb493c8-641f-4845-974f-68e688637130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:34.563344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:34.565844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:34.566124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:34.566295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] PATH_INFO: `/v3/users/4dadf566d4b14f4a84920dd441831022` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:34.578491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dadf566d4b14f4a84920dd441831022)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:34.586416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:34.610124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:34.801078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 876/1736] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:34.819102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb493c8-641f-4845-974f-68e688637130 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:34.871189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:34.942091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb493c8-641f-4845-974f-68e688637130 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:34.946650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634dab2-2465-472f-a09f-03766b93aba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:34.959325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634dab2-2465-472f-a09f-03766b93aba3 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:34.959950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634dab2-2465-472f-a09f-03766b93aba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:34.960334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634dab2-2465-472f-a09f-03766b93aba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:34.973329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dadf566d4b14f4a84920dd441831022'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:34.976168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dadf566d4b14f4a84920dd441831022'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:34.978089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634dab2-2465-472f-a09f-03766b93aba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:34.980597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634dab2-2465-472f-a09f-03766b93aba3 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:34.996733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 877/1737] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:35.016408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dadf566d4b14f4a84920dd441831022'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:35.053257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dadf566d4b14f4a84920dd441831022'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:35.187711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dadf566d4b14f4a84920dd441831022'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:35.211807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 861/1738] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:34 2022] DELETE /identity/v3/users/4dadf566d4b14f4a84920dd441831022 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:35.216956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9eecce-7ca5-4aef-b434-0fec15c3981a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:35.217331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9eecce-7ca5-4aef-b434-0fec15c3981a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:35.217784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9eecce-7ca5-4aef-b434-0fec15c3981a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:35.278352 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6d9eecce-7ca5-4aef-b434-0fec15c3981a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:35.279619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6d9eecce-7ca5-4aef-b434-0fec15c3981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:35.416069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 878/1739] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:35.436741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:35.438992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:35.571934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:35.617846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05f0fd-8b5a-4a37-81a4-af855865a76b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:35.642788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4480bc8-27e3-44b3-9fe4-b8df6462d2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:35.643464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4480bc8-27e3-44b3-9fe4-b8df6462d2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:35.643739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4480bc8-27e3-44b3-9fe4-b8df6462d2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:35.644022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4480bc8-27e3-44b3-9fe4-b8df6462d2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:35.665619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4480bc8-27e3-44b3-9fe4-b8df6462d2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:35.667710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4480bc8-27e3-44b3-9fe4-b8df6462d2c7 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:35.743886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 862/1740] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:37.068488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9eecce-7ca5-4aef-b434-0fec15c3981a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:37.097387 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6d9eecce-7ca5-4aef-b434-0fec15c3981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:37.132715 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9eecce-7ca5-4aef-b434-0fec15c3981a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:37.140444 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:37.141688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:37.142184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:37.142581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] PATH_INFO: `/v3/projects/710f8018e9e94e8db41725502dce8978` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:37.144575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=710f8018e9e94e8db41725502dce8978)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:37.147119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:37.319550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6eab2-53b3-4a12-ba90-1f59ebe499eb None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:37.320125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6eab2-53b3-4a12-ba90-1f59ebe499eb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:37.320202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6eab2-53b3-4a12-ba90-1f59ebe499eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:37.388181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f5e6eab2-53b3-4a12-ba90-1f59ebe499eb None None] MFA Rules not processed for user `5a304d58eff84a87953dc7af6a0afe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:37.393497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f5e6eab2-53b3-4a12-ba90-1f59ebe499eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:37.472445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '710f8018e9e94e8db41725502dce8978', 'request_id': 'req-188a42e9-bea6-4eb6-aba1-6ac5056833c1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:37.488698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '710f8018e9e94e8db41725502dce8978', 'request_id': 'req-188a42e9-bea6-4eb6-aba1-6ac5056833c1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:37.567768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '710f8018e9e94e8db41725502dce8978', 'request_id': 'req-188a42e9-bea6-4eb6-aba1-6ac5056833c1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:37.599899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] The token cache is being invalidate because project 710f8018e9e94e8db41725502dce8978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:37.605223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:37.605223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:37.610698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 879/1741] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:37 2022] DELETE /identity/v3/projects/710f8018e9e94e8db41725502dce8978 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:37.611883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:37.617406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:37.661468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 863/1742] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:37.665074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae4a6-2bff-457d-ba4e-86bca7272f6f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:37.665373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae4a6-2bff-457d-ba4e-86bca7272f6f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:37.665655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae4a6-2bff-457d-ba4e-86bca7272f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:37.763033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c2eae4a6-2bff-457d-ba4e-86bca7272f6f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:37.767799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c2eae4a6-2bff-457d-ba4e-86bca7272f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:37.796631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:37.820226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a42e9-bea6-4eb6-aba1-6ac5056833c1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:37.834722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3212cf-c011-4b3f-8f10-ae7dcfbe3988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:37.836396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3212cf-c011-4b3f-8f10-ae7dcfbe3988 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:37.840638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3212cf-c011-4b3f-8f10-ae7dcfbe3988 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:37.840638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3212cf-c011-4b3f-8f10-ae7dcfbe3988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:37.848845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2d3212cf-c011-4b3f-8f10-ae7dcfbe3988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:37.970097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 864/1743] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:37.979666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9b9c-359e-4a64-96a6-4197824b0e16 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:37.980014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9b9c-359e-4a64-96a6-4197824b0e16 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:37.980780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa9b9c-359e-4a64-96a6-4197824b0e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:38.033147 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c7aa9b9c-359e-4a64-96a6-4197824b0e16 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:38.042265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c7aa9b9c-359e-4a64-96a6-4197824b0e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:38.090316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 865/1744] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:38.094309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9b9c-359e-4a64-96a6-4197824b0e16 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:38.155109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa9b9c-359e-4a64-96a6-4197824b0e16 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:38.174906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64c967-270a-46ca-9d1c-1b02ac072c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:38.176814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64c967-270a-46ca-9d1c-1b02ac072c2d admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:38.193828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3212cf-c011-4b3f-8f10-ae7dcfbe3988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:38.196117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3212cf-c011-4b3f-8f10-ae7dcfbe3988 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:38.208418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64c967-270a-46ca-9d1c-1b02ac072c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:38.209335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64c967-270a-46ca-9d1c-1b02ac072c2d admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:38.210171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64c967-270a-46ca-9d1c-1b02ac072c2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:38.210488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64c967-270a-46ca-9d1c-1b02ac072c2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:38.221106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64c967-270a-46ca-9d1c-1b02ac072c2d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:38.245396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 880/1745] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:38.272055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 866/1746] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:38.276394 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3212cf-c011-4b3f-8f10-ae7dcfbe3988 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:38.294172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3212cf-c011-4b3f-8f10-ae7dcfbe3988 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:38.308982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41c4ea-8ac0-46c9-957a-179990fbb363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:38.309234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41c4ea-8ac0-46c9-957a-179990fbb363 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:38.309521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41c4ea-8ac0-46c9-957a-179990fbb363 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:38.312701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41c4ea-8ac0-46c9-957a-179990fbb363 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:38.312701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41c4ea-8ac0-46c9-957a-179990fbb363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:38.322185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41c4ea-8ac0-46c9-957a-179990fbb363 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:38.382483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:38.477296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 881/1747] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:38 2022] POST /identity/v3/projects => generated 405 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:58:38.481541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64c967-270a-46ca-9d1c-1b02ac072c2d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:38.535946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64c967-270a-46ca-9d1c-1b02ac072c2d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:38.677555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad45e5b-8858-4093-9e3a-89aa12b05599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:38.678503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad45e5b-8858-4093-9e3a-89aa12b05599 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:38.678791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad45e5b-8858-4093-9e3a-89aa12b05599 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:38.679065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad45e5b-8858-4093-9e3a-89aa12b05599 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:38.679794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad45e5b-8858-4093-9e3a-89aa12b05599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:38.696256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad45e5b-8858-4093-9e3a-89aa12b05599 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:38.843414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 867/1748] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:38 2022] POST /identity/v3/users => generated 303 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:38.843657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41c4ea-8ac0-46c9-957a-179990fbb363 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:38.866788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41c4ea-8ac0-46c9-957a-179990fbb363 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:38.872427 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9ba9-c61a-4540-9d4b-2db93048c9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:38.873290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9ba9-c61a-4540-9d4b-2db93048c9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:38.876691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9ba9-c61a-4540-9d4b-2db93048c9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:38.876691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9ba9-c61a-4540-9d4b-2db93048c9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:38.878435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9ba9-c61a-4540-9d4b-2db93048c9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:38.880081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9ba9-c61a-4540-9d4b-2db93048c9d8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:38.913836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad45e5b-8858-4093-9e3a-89aa12b05599 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:38.916233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 882/1749] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:38.928746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad45e5b-8858-4093-9e3a-89aa12b05599 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:38.936336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da97bdb-0ef7-4b45-852f-8feddc859fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:38.937483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97bdb-0ef7-4b45-852f-8feddc859fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:38.937920 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97bdb-0ef7-4b45-852f-8feddc859fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:38.938292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97bdb-0ef7-4b45-852f-8feddc859fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:38.939389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da97bdb-0ef7-4b45-852f-8feddc859fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:38.941268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da97bdb-0ef7-4b45-852f-8feddc859fc9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:39.019751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 868/1750] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:58:39.021774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9ba9-c61a-4540-9d4b-2db93048c9d8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:39.068142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9ba9-c61a-4540-9d4b-2db93048c9d8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:39.074630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f4c68-977a-43ed-a24e-cab5108d7fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:39.076211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f4c68-977a-43ed-a24e-cab5108d7fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:39.076998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f4c68-977a-43ed-a24e-cab5108d7fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:39.077456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f4c68-977a-43ed-a24e-cab5108d7fb2 admin admin] PATH_INFO: `/v3/projects/b11d7aef4f3e4ed595525333b0a9aac8/users/c2aba30a366846a7a53c5aba2da207cc/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:39.100495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f4c68-977a-43ed-a24e-cab5108d7fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11d7aef4f3e4ed595525333b0a9aac8, user_id=c2aba30a366846a7a53c5aba2da207cc, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:39.125294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f4c68-977a-43ed-a24e-cab5108d7fb2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:39.243655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 883/1751] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:39 2022] PUT /identity/v3/projects/b11d7aef4f3e4ed595525333b0a9aac8/users/c2aba30a366846a7a53c5aba2da207cc/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:39.244501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e707a8-3447-4d64-8831-43ce5a3b7642 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:39.250549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e707a8-3447-4d64-8831-43ce5a3b7642 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:39.250549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e707a8-3447-4d64-8831-43ce5a3b7642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:39.285109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a8e707a8-3447-4d64-8831-43ce5a3b7642 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:39.285109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a8e707a8-3447-4d64-8831-43ce5a3b7642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:39.383977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 869/1752] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:39.394356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f4c68-977a-43ed-a24e-cab5108d7fb2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:39.396248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-554f4c68-977a-43ed-a24e-cab5108d7fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:39.600789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f4c68-977a-43ed-a24e-cab5108d7fb2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:39.615515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb7440-ac2e-46d2-b01b-66d3f94ff776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:39.616210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb7440-ac2e-46d2-b01b-66d3f94ff776 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:39.616518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb7440-ac2e-46d2-b01b-66d3f94ff776 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:39.616804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb7440-ac2e-46d2-b01b-66d3f94ff776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:39.634417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb7440-ac2e-46d2-b01b-66d3f94ff776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:39.636523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb7440-ac2e-46d2-b01b-66d3f94ff776 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:39.656978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 884/1753] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:41.944765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e707a8-3447-4d64-8831-43ce5a3b7642 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:41.946303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a8e707a8-3447-4d64-8831-43ce5a3b7642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:41.958903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e707a8-3447-4d64-8831-43ce5a3b7642 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:41.964029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:41.964523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:41.964809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:41.965090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] PATH_INFO: `/v3/users/b9fae63789d341c3996f8066dca8dfe9` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:41.966320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9fae63789d341c3996f8066dca8dfe9)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:41.968297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:41.970788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:42.110669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2d96-c237-43a0-8bf9-e9b96686f478 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:42.111272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2d96-c237-43a0-8bf9-e9b96686f478 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:42.111785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f2d96-c237-43a0-8bf9-e9b96686f478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:42.161618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e49f2d96-c237-43a0-8bf9-e9b96686f478 None None] MFA Rules not processed for user `c2aba30a366846a7a53c5aba2da207cc`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:42.165296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e49f2d96-c237-43a0-8bf9-e9b96686f478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:42.227223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9fae63789d341c3996f8066dca8dfe9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:42.228181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9fae63789d341c3996f8066dca8dfe9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:42.250027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9fae63789d341c3996f8066dca8dfe9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:42.252537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9fae63789d341c3996f8066dca8dfe9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:42.256387 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 885/1754] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:42 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:42.260642 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f2d96-c237-43a0-8bf9-e9b96686f478 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:42.262152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e49f2d96-c237-43a0-8bf9-e9b96686f478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:42.275684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f2d96-c237-43a0-8bf9-e9b96686f478 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:42.281152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395873f4-eba6-4eab-853a-c059484a27b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:42.284617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395873f4-eba6-4eab-853a-c059484a27b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:42.284617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395873f4-eba6-4eab-853a-c059484a27b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:42.284617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395873f4-eba6-4eab-853a-c059484a27b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:42.284617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395873f4-eba6-4eab-853a-c059484a27b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:42.285146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395873f4-eba6-4eab-853a-c059484a27b5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:42.342579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-50425045-9775-4927-8aae-61e66fb64bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9fae63789d341c3996f8066dca8dfe9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:42.349856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 870/1755] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:41 2022] DELETE /identity/v3/users/b9fae63789d341c3996f8066dca8dfe9 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:42.355338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56503ffb-4a3d-4f89-9393-4ef380eba992 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:42.355794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56503ffb-4a3d-4f89-9393-4ef380eba992 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:42.356100 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56503ffb-4a3d-4f89-9393-4ef380eba992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:42.372727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-56503ffb-4a3d-4f89-9393-4ef380eba992 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:42.373942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-56503ffb-4a3d-4f89-9393-4ef380eba992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:42.433229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 871/1756] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:58:42.445839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 886/1757] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:42 2022] POST /identity/v3/projects => generated 403 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:58:42.448610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56503ffb-4a3d-4f89-9393-4ef380eba992 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:42.449739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395873f4-eba6-4eab-853a-c059484a27b5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:42.450144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-56503ffb-4a3d-4f89-9393-4ef380eba992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:42.490268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:42.529559 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395873f4-eba6-4eab-853a-c059484a27b5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:42.529771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56503ffb-4a3d-4f89-9393-4ef380eba992 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:42.537372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33d542-d785-464d-9a9d-27583318b6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:42.537372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33d542-d785-464d-9a9d-27583318b6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:42.537372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33d542-d785-464d-9a9d-27583318b6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:42.537372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33d542-d785-464d-9a9d-27583318b6d2 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:42.537820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33d542-d785-464d-9a9d-27583318b6d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:42.540187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33d542-d785-464d-9a9d-27583318b6d2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:42.602959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5109b-4c6c-4c18-af89-c7be569897d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:42.603681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5109b-4c6c-4c18-af89-c7be569897d6 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:42.604023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5109b-4c6c-4c18-af89-c7be569897d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:42.604392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5109b-4c6c-4c18-af89-c7be569897d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:42.616221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5109b-4c6c-4c18-af89-c7be569897d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:42.617762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5109b-4c6c-4c18-af89-c7be569897d6 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:42.628314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 872/1758] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:58:42.661465 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 887/1759] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:42 2022] POST /identity/v3/users => generated 308 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:58:42.670214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5109b-4c6c-4c18-af89-c7be569897d6 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:42.694287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5109b-4c6c-4c18-af89-c7be569897d6 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:42.703618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd7bc5-15c1-48dd-a3a6-854c9d5c2952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:42.704397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd7bc5-15c1-48dd-a3a6-854c9d5c2952 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:42.704747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd7bc5-15c1-48dd-a3a6-854c9d5c2952 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:42.705239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd7bc5-15c1-48dd-a3a6-854c9d5c2952 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:42.706251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd7bc5-15c1-48dd-a3a6-854c9d5c2952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:42.712198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd7bc5-15c1-48dd-a3a6-854c9d5c2952 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:42.724798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 873/1760] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:42.730863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33d542-d785-464d-9a9d-27583318b6d2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:42.743421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33d542-d785-464d-9a9d-27583318b6d2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:42.749694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c5f57-f2fe-4556-af46-5240c303ca52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:42.750555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5f57-f2fe-4556-af46-5240c303ca52 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:42.751058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5f57-f2fe-4556-af46-5240c303ca52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:42.751437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5f57-f2fe-4556-af46-5240c303ca52 admin admin] PATH_INFO: `/v3/projects/a4d2175740794d35962a282bb41589dc/users/b2d2a4a7210241908cf956607da8a833/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:42.769922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c5f57-f2fe-4556-af46-5240c303ca52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d2175740794d35962a282bb41589dc, user_id=b2d2a4a7210241908cf956607da8a833, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:42.774341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c5f57-f2fe-4556-af46-5240c303ca52 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:42.875896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 888/1761] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:42 2022] PUT /identity/v3/projects/a4d2175740794d35962a282bb41589dc/users/b2d2a4a7210241908cf956607da8a833/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:42.876934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd7bc5-15c1-48dd-a3a6-854c9d5c2952 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:42.889017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd7bc5-15c1-48dd-a3a6-854c9d5c2952 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:43.029297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10bf8e-b391-45b5-9727-0c80aa383b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:43.030123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10bf8e-b391-45b5-9727-0c80aa383b2a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:43.030633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10bf8e-b391-45b5-9727-0c80aa383b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:43.031528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10bf8e-b391-45b5-9727-0c80aa383b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:43.032656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10bf8e-b391-45b5-9727-0c80aa383b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:43.034585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10bf8e-b391-45b5-9727-0c80aa383b2a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:43.069670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 874/1762] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:43.073078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c5f57-f2fe-4556-af46-5240c303ca52 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:43.107174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c5f57-f2fe-4556-af46-5240c303ca52 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:43.117801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbfbdc-8eee-4ec2-9b6e-4d5d5498e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:43.119531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbfbdc-8eee-4ec2-9b6e-4d5d5498e99d admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:43.119979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbfbdc-8eee-4ec2-9b6e-4d5d5498e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:43.121556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbfbdc-8eee-4ec2-9b6e-4d5d5498e99d admin admin] PATH_INFO: `/v3/projects/a4d2175740794d35962a282bb41589dc/users/b2d2a4a7210241908cf956607da8a833/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:43.124181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbfbdc-8eee-4ec2-9b6e-4d5d5498e99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d2175740794d35962a282bb41589dc, user_id=b2d2a4a7210241908cf956607da8a833, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:43.127098 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbfbdc-8eee-4ec2-9b6e-4d5d5498e99d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:43.145781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 889/1763] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:43 2022] PUT /identity/v3/projects/a4d2175740794d35962a282bb41589dc/users/b2d2a4a7210241908cf956607da8a833/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:43.227085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10bf8e-b391-45b5-9727-0c80aa383b2a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:43.231167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5e10bf8e-b391-45b5-9727-0c80aa383b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:43.268424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10bf8e-b391-45b5-9727-0c80aa383b2a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:43.369481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:43.370776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:43.374007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:43.374007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] PATH_INFO: `/v3/projects/8c58d6b0730c4c4cb619a7774d416b3d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:43.374007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c58d6b0730c4c4cb619a7774d416b3d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:43.376324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:43.551769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c58d6b0730c4c4cb619a7774d416b3d', 'request_id': 'req-0e742ac7-9823-4da4-82b4-0be5255fe992'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:43.593695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c58d6b0730c4c4cb619a7774d416b3d', 'request_id': 'req-0e742ac7-9823-4da4-82b4-0be5255fe992'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:43.794800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c58d6b0730c4c4cb619a7774d416b3d', 'request_id': 'req-0e742ac7-9823-4da4-82b4-0be5255fe992'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:43.838528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] The token cache is being invalidate because project 8c58d6b0730c4c4cb619a7774d416b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:43.840591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:43.841586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0e742ac7-9823-4da4-82b4-0be5255fe992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:43.847229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 875/1764] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:43 2022] DELETE /identity/v3/projects/8c58d6b0730c4c4cb619a7774d416b3d => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:43.847863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbfbdc-8eee-4ec2-9b6e-4d5d5498e99d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:43.850864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-64fbfbdc-8eee-4ec2-9b6e-4d5d5498e99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:43.875359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbfbdc-8eee-4ec2-9b6e-4d5d5498e99d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:43.997801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:43.997801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:43.997801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:43.997801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] PATH_INFO: `/v3/users/1a7c8c24ef1f43889aa5d7f3bc556ae0` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:43.998191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a7c8c24ef1f43889aa5d7f3bc556ae0)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:44.000069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:44.389635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a7c8c24ef1f43889aa5d7f3bc556ae0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:44.394659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a7c8c24ef1f43889aa5d7f3bc556ae0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:44.414251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a7c8c24ef1f43889aa5d7f3bc556ae0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:44.430964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a7c8c24ef1f43889aa5d7f3bc556ae0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:44.491286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:44.550352 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a7c8c24ef1f43889aa5d7f3bc556ae0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:44.561385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 890/1765] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:43 2022] DELETE /identity/v3/users/1a7c8c24ef1f43889aa5d7f3bc556ae0 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:58:44.874666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcb758-71a1-4990-91af-20a40a2d26e9 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:44.875054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcb758-71a1-4990-91af-20a40a2d26e9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:44.875430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcb758-71a1-4990-91af-20a40a2d26e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:44.931922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9cdcb758-71a1-4990-91af-20a40a2d26e9 None None] MFA Rules not processed for user `b2d2a4a7210241908cf956607da8a833`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:44.936566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9cdcb758-71a1-4990-91af-20a40a2d26e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:45.128953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 876/1766] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:45.408172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:45.410542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:45.417749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:45.545999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ae299-260e-49e0-8f86-fc04721e3706 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:45.560363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550e26e-fec3-4106-be3e-4db688d104cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:45.561189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550e26e-fec3-4106-be3e-4db688d104cd service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:45.561581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550e26e-fec3-4106-be3e-4db688d104cd service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:45.561995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550e26e-fec3-4106-be3e-4db688d104cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:45.568565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2550e26e-fec3-4106-be3e-4db688d104cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:45.658095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550e26e-fec3-4106-be3e-4db688d104cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:45.659698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550e26e-fec3-4106-be3e-4db688d104cd service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:45.673247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 891/1767] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:46.891371 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819aaa-d81a-4703-af84-8969b5ce25d2 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:46.892330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819aaa-d81a-4703-af84-8969b5ce25d2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:46.892500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819aaa-d81a-4703-af84-8969b5ce25d2 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:46.894711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 877/1768] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 09:58:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:58:46.905858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0173b-a928-4e39-8c56-3d3074029b8e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:46.906650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0173b-a928-4e39-8c56-3d3074029b8e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:46.906956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0173b-a928-4e39-8c56-3d3074029b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:46.931919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5aa0173b-a928-4e39-8c56-3d3074029b8e None None] MFA Rules not processed for user `566c0c4bdcaf4c13b060af4bbff5bc04`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:46.934420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5aa0173b-a928-4e39-8c56-3d3074029b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:46.970045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4819aaa-d81a-4703-af84-8969b5ce25d2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:46.972175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d4819aaa-d81a-4703-af84-8969b5ce25d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:47.033053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 892/1769] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 09:58:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:47.051805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0173b-a928-4e39-8c56-3d3074029b8e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:47.058771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5aa0173b-a928-4e39-8c56-3d3074029b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:47.092409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0173b-a928-4e39-8c56-3d3074029b8e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:47.109853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e626-7e17-425a-8365-24dbc6ae0b12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:47.110567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e626-7e17-425a-8365-24dbc6ae0b12 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:47.110929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e626-7e17-425a-8365-24dbc6ae0b12 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:47.111127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e626-7e17-425a-8365-24dbc6ae0b12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:47.116193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4819aaa-d81a-4703-af84-8969b5ce25d2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:47.126729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:47.127635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:47.133036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297e626-7e17-425a-8365-24dbc6ae0b12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:47.133634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:47.133634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] PATH_INFO: `/v3/projects/3ce71271b10842a3aeaa7f2264d20828` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:47.133784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297e626-7e17-425a-8365-24dbc6ae0b12 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:47.138898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce71271b10842a3aeaa7f2264d20828)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:47.140907 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:47.144176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 893/1770] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:47.342534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce71271b10842a3aeaa7f2264d20828', 'request_id': 'req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:47.358082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce71271b10842a3aeaa7f2264d20828', 'request_id': 'req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:47.465436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce71271b10842a3aeaa7f2264d20828', 'request_id': 'req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:47.471143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a0bb5-5573-4500-aa1f-6652e16c8630 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:47.471835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a0bb5-5573-4500-aa1f-6652e16c8630 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:47.472832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a0bb5-5573-4500-aa1f-6652e16c8630 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:47.478712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 894/1771] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 09:58:47 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 09:58:47.488176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] The token cache is being invalidate because project 3ce71271b10842a3aeaa7f2264d20828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:47.488478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:47.489169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fe28943d-b0a6-41ec-85a5-9e6dc86ab0e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:47.494725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 878/1772] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:46 2022] DELETE /identity/v3/projects/3ce71271b10842a3aeaa7f2264d20828 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:58:47.496863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17ac4d-851f-4ec2-86e9-6711b0817ca1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:47.497152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17ac4d-851f-4ec2-86e9-6711b0817ca1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:47.497433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17ac4d-851f-4ec2-86e9-6711b0817ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:47.508114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3707d-8da2-434c-9f6b-e4715ac65ab6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:47.508446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3707d-8da2-434c-9f6b-e4715ac65ab6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:47.511232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3707d-8da2-434c-9f6b-e4715ac65ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:47.527130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0c17ac4d-851f-4ec2-86e9-6711b0817ca1 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:47.528857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0c17ac4d-851f-4ec2-86e9-6711b0817ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:47.543054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7bd3707d-8da2-434c-9f6b-e4715ac65ab6 None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:47.547760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7bd3707d-8da2-434c-9f6b-e4715ac65ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:47.704023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 895/1773] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:47.705554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f2807-afb1-45cb-afce-675ad5e530cb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:47.705852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f2807-afb1-45cb-afce-675ad5e530cb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:47.706162 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f2807-afb1-45cb-afce-675ad5e530cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:47.751380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 879/1774] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 09:58:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:47.768287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2e6f2807-afb1-45cb-afce-675ad5e530cb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:47.769565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2e6f2807-afb1-45cb-afce-675ad5e530cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:47.782319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 896/1775] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:47.786452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3707d-8da2-434c-9f6b-e4715ac65ab6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:47.798188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3707d-8da2-434c-9f6b-e4715ac65ab6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:47.803091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:47.803796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:47.804091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:47.806615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:47.806615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:47.806615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:47.807242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:47.819532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 880/1776] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:58:47.824448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f2807-afb1-45cb-afce-675ad5e530cb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:47.851230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f2807-afb1-45cb-afce-675ad5e530cb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:47.857149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a064ef9-c8bc-4ba0-9eb3-15c83d9cc72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:47.859756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a064ef9-c8bc-4ba0-9eb3-15c83d9cc72d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:47.860026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a064ef9-c8bc-4ba0-9eb3-15c83d9cc72d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:47.861330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a064ef9-c8bc-4ba0-9eb3-15c83d9cc72d admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:47.862124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a064ef9-c8bc-4ba0-9eb3-15c83d9cc72d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:47.868061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a064ef9-c8bc-4ba0-9eb3-15c83d9cc72d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:47.871581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:47.874512 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:47.898520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f17d1b-1ea4-405d-ab72-1d4e3ab7b2a9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:47.906548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662de9b-4958-453d-aac9-9031524af8e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:47.907380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662de9b-4958-453d-aac9-9031524af8e5 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:47.907832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662de9b-4958-453d-aac9-9031524af8e5 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:47.908117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662de9b-4958-453d-aac9-9031524af8e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:47.922403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662de9b-4958-453d-aac9-9031524af8e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:47.924162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662de9b-4958-453d-aac9-9031524af8e5 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:47.935432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 881/1777] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:58:48.010193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 897/1778] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:47 2022] POST /identity/v3/projects => generated 393 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:48.015500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662de9b-4958-453d-aac9-9031524af8e5 service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:48.026982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662de9b-4958-453d-aac9-9031524af8e5 service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:48.106846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a064ef9-c8bc-4ba0-9eb3-15c83d9cc72d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:48.109182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8a064ef9-c8bc-4ba0-9eb3-15c83d9cc72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:48.129476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f7a01-a7df-406c-869a-5d912ae405c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:48.130327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f7a01-a7df-406c-869a-5d912ae405c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:48.130617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f7a01-a7df-406c-869a-5d912ae405c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:48.136293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f7a01-a7df-406c-869a-5d912ae405c9 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:48.136293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f7a01-a7df-406c-869a-5d912ae405c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:48.136293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f7a01-a7df-406c-869a-5d912ae405c9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:48.238361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a064ef9-c8bc-4ba0-9eb3-15c83d9cc72d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:48.249877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:48.250804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:48.251685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:48.252307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] PATH_INFO: `/v3/users/c2aba30a366846a7a53c5aba2da207cc` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:48.266780 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2aba30a366846a7a53c5aba2da207cc)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:48.270719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:48.298264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 882/1779] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:48 2022] POST /identity/v3/users => generated 297 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:48.302582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f7a01-a7df-406c-869a-5d912ae405c9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:48.327188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f7a01-a7df-406c-869a-5d912ae405c9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:48.337685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123aa0b-7020-44c3-a5ae-2fb0518ff361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:48.338442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123aa0b-7020-44c3-a5ae-2fb0518ff361 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:48.338761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123aa0b-7020-44c3-a5ae-2fb0518ff361 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:48.339048 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123aa0b-7020-44c3-a5ae-2fb0518ff361 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:48.339901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123aa0b-7020-44c3-a5ae-2fb0518ff361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:48.341975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123aa0b-7020-44c3-a5ae-2fb0518ff361 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:48.363642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 883/1780] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:48.369584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123aa0b-7020-44c3-a5ae-2fb0518ff361 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:48.384657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123aa0b-7020-44c3-a5ae-2fb0518ff361 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:48.391835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db46f68-908d-4e73-9e9b-5911e5f67c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:48.392621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db46f68-908d-4e73-9e9b-5911e5f67c40 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:48.392996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db46f68-908d-4e73-9e9b-5911e5f67c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:48.393292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db46f68-908d-4e73-9e9b-5911e5f67c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:48.394104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db46f68-908d-4e73-9e9b-5911e5f67c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:48.395830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db46f68-908d-4e73-9e9b-5911e5f67c40 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:48.413243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 884/1781] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:58:48.416766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:48.418142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db46f68-908d-4e73-9e9b-5911e5f67c40 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:48.448676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db46f68-908d-4e73-9e9b-5911e5f67c40 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:48.454006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f597b1-ea5b-4780-8b2e-a863402e6be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:48.454913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f597b1-ea5b-4780-8b2e-a863402e6be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:48.455283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f597b1-ea5b-4780-8b2e-a863402e6be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:48.455721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f597b1-ea5b-4780-8b2e-a863402e6be4 admin admin] PATH_INFO: `/v3/projects/c545774a9d6c427fa49f554b5a3e7b51/users/cee47690954c4112a92a4e9d95172bd9/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:48.490871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f597b1-ea5b-4780-8b2e-a863402e6be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c545774a9d6c427fa49f554b5a3e7b51, user_id=cee47690954c4112a92a4e9d95172bd9, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:48.493748 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f597b1-ea5b-4780-8b2e-a863402e6be4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:48.611918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2aba30a366846a7a53c5aba2da207cc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:48.612583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2aba30a366846a7a53c5aba2da207cc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:48.620857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2aba30a366846a7a53c5aba2da207cc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:48.633497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2aba30a366846a7a53c5aba2da207cc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:48.674306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 885/1782] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:48 2022] PUT /identity/v3/projects/c545774a9d6c427fa49f554b5a3e7b51/users/cee47690954c4112a92a4e9d95172bd9/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:48.685433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7832ca-9fc6-4eab-a964-9d167fc2c6a0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:48.685433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7832ca-9fc6-4eab-a964-9d167fc2c6a0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:48.685644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7832ca-9fc6-4eab-a964-9d167fc2c6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:48.738723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0b7832ca-9fc6-4eab-a964-9d167fc2c6a0 None None] MFA Rules not processed for user `cee47690954c4112a92a4e9d95172bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:48.739986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0b7832ca-9fc6-4eab-a964-9d167fc2c6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:48.742852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2aba30a366846a7a53c5aba2da207cc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:48.750370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 898/1783] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:48 2022] DELETE /identity/v3/users/c2aba30a366846a7a53c5aba2da207cc => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:58:48.877575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:48.878159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 886/1784] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:58:48.879179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:48.992808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be99027-d395-431c-931b-77f7675c690b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:49.030287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d466bd1-49e2-4b83-b0a2-2df17a129126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:49.036452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d466bd1-49e2-4b83-b0a2-2df17a129126 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:49.037033 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d466bd1-49e2-4b83-b0a2-2df17a129126 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:49.037532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d466bd1-49e2-4b83-b0a2-2df17a129126 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:49.038521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d466bd1-49e2-4b83-b0a2-2df17a129126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:49.045825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d466bd1-49e2-4b83-b0a2-2df17a129126 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:49.178438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7832ca-9fc6-4eab-a964-9d167fc2c6a0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:49.180582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0b7832ca-9fc6-4eab-a964-9d167fc2c6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:49.181094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 899/1785] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:48 2022] POST /identity/v3/projects => generated 393 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:49.185035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d466bd1-49e2-4b83-b0a2-2df17a129126 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:49.289966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d466bd1-49e2-4b83-b0a2-2df17a129126 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:49.295873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6bb3b-2f0b-4a50-a349-c401c9b7d886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:49.296562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6bb3b-2f0b-4a50-a349-c401c9b7d886 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:49.296849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6bb3b-2f0b-4a50-a349-c401c9b7d886 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:49.297181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6bb3b-2f0b-4a50-a349-c401c9b7d886 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:49.298021 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6bb3b-2f0b-4a50-a349-c401c9b7d886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:49.300338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6bb3b-2f0b-4a50-a349-c401c9b7d886 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:49.326565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:49.329115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7832ca-9fc6-4eab-a964-9d167fc2c6a0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:49.337722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:49.339150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:49.339618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:49.340075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] PATH_INFO: `/v3/projects/b11d7aef4f3e4ed595525333b0a9aac8` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:49.342771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b11d7aef4f3e4ed595525333b0a9aac8)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:49.346114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:49.449229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 900/1786] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:49 2022] POST /identity/v3/users => generated 303 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:58:49.453555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6bb3b-2f0b-4a50-a349-c401c9b7d886 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:49.469157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6bb3b-2f0b-4a50-a349-c401c9b7d886 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:49.472516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa805ed0-fa5d-455c-a9a8-d3e533c70462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:49.473684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805ed0-fa5d-455c-a9a8-d3e533c70462 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:49.474380 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805ed0-fa5d-455c-a9a8-d3e533c70462 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:49.475310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805ed0-fa5d-455c-a9a8-d3e533c70462 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:49.476691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa805ed0-fa5d-455c-a9a8-d3e533c70462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:49.478693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa805ed0-fa5d-455c-a9a8-d3e533c70462 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:49.499391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 901/1787] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:58:49.503369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa805ed0-fa5d-455c-a9a8-d3e533c70462 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:49.517451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa805ed0-fa5d-455c-a9a8-d3e533c70462 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:49.536855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b11d7aef4f3e4ed595525333b0a9aac8', 'request_id': 'req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:49.543238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b11d7aef4f3e4ed595525333b0a9aac8', 'request_id': 'req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:49.595372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cda56-c84c-4606-90d1-7498bebaa901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:49.596279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cda56-c84c-4606-90d1-7498bebaa901 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:49.596568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cda56-c84c-4606-90d1-7498bebaa901 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:49.596852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cda56-c84c-4606-90d1-7498bebaa901 admin admin] PATH_INFO: `/v3/projects/92a2eba00ed04b9fb625c3b718c3fafd/users/13560b4b5fb94b068bb7f84af8ec57b0/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:49.612543 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cda56-c84c-4606-90d1-7498bebaa901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a2eba00ed04b9fb625c3b718c3fafd, user_id=13560b4b5fb94b068bb7f84af8ec57b0, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:49.615280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cda56-c84c-4606-90d1-7498bebaa901 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:49.639730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b11d7aef4f3e4ed595525333b0a9aac8', 'request_id': 'req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:49.652123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] The token cache is being invalidate because project b11d7aef4f3e4ed595525333b0a9aac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:49.652406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:49.653043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:49.654720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 887/1788] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:49 2022] DELETE /identity/v3/projects/b11d7aef4f3e4ed595525333b0a9aac8 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:58:49.658578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:49.659892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:49.672193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a31e6-4c4d-42ca-af2d-d4c4f3a8af37 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:49.677410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:49.678377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:49.678784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:49.679058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] PATH_INFO: `/v3/users/b2d2a4a7210241908cf956607da8a833` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:49.680819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d2a4a7210241908cf956607da8a833)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:49.685181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:49.690604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 902/1789] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:49 2022] PUT /identity/v3/projects/92a2eba00ed04b9fb625c3b718c3fafd/users/13560b4b5fb94b068bb7f84af8ec57b0/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:58:49.694573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cda56-c84c-4606-90d1-7498bebaa901 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:49.696305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-441cda56-c84c-4606-90d1-7498bebaa901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:49.782492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cda56-c84c-4606-90d1-7498bebaa901 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:49.787095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b47595-81f3-4ba5-81ac-0bcd7379be37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:49.787890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b47595-81f3-4ba5-81ac-0bcd7379be37 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:49.788235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b47595-81f3-4ba5-81ac-0bcd7379be37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:49.788594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b47595-81f3-4ba5-81ac-0bcd7379be37 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:49.789424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b47595-81f3-4ba5-81ac-0bcd7379be37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:49.791160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b47595-81f3-4ba5-81ac-0bcd7379be37 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:49.801350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 903/1790] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:58:49.805552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b47595-81f3-4ba5-81ac-0bcd7379be37 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:49.817742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b47595-81f3-4ba5-81ac-0bcd7379be37 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:49.822513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef22786-7cd7-42ab-8e45-cf8a145a2c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:49.824655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef22786-7cd7-42ab-8e45-cf8a145a2c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:49.824948 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef22786-7cd7-42ab-8e45-cf8a145a2c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:49.825229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef22786-7cd7-42ab-8e45-cf8a145a2c9e admin admin] PATH_INFO: `/v3/projects/92a2eba00ed04b9fb625c3b718c3fafd/users/13560b4b5fb94b068bb7f84af8ec57b0/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:49.830596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef22786-7cd7-42ab-8e45-cf8a145a2c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a2eba00ed04b9fb625c3b718c3fafd, user_id=13560b4b5fb94b068bb7f84af8ec57b0, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:49.833702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef22786-7cd7-42ab-8e45-cf8a145a2c9e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:49.844704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 904/1791] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:49 2022] PUT /identity/v3/projects/92a2eba00ed04b9fb625c3b718c3fafd/users/13560b4b5fb94b068bb7f84af8ec57b0/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:58:49.855513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3293e9-97a4-4b94-b531-0befc3a8bbec None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:49.856464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3293e9-97a4-4b94-b531-0befc3a8bbec None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:49.856748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3293e9-97a4-4b94-b531-0befc3a8bbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:49.894582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9e3293e9-97a4-4b94-b531-0befc3a8bbec None None] MFA Rules not processed for user `13560b4b5fb94b068bb7f84af8ec57b0`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:49.895825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9e3293e9-97a4-4b94-b531-0befc3a8bbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:49.943648 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d2a4a7210241908cf956607da8a833'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:49.944597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d2a4a7210241908cf956607da8a833'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:49.957678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d2a4a7210241908cf956607da8a833'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:49.964867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d2a4a7210241908cf956607da8a833'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:49.967029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 905/1792] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:49.978586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3293e9-97a4-4b94-b531-0befc3a8bbec None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:49.982055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9e3293e9-97a4-4b94-b531-0befc3a8bbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:50.030791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d2a4a7210241908cf956607da8a833'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:50.037793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3293e9-97a4-4b94-b531-0befc3a8bbec None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:50.041110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 888/1793] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:49 2022] DELETE /identity/v3/users/b2d2a4a7210241908cf956607da8a833 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:50.111408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04b728-f4f0-4532-92e8-e0e24cb98c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:50.112101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04b728-f4f0-4532-92e8-e0e24cb98c8e service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:50.112389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04b728-f4f0-4532-92e8-e0e24cb98c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:50.112672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04b728-f4f0-4532-92e8-e0e24cb98c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:50.234113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04b728-f4f0-4532-92e8-e0e24cb98c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:50.235604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04b728-f4f0-4532-92e8-e0e24cb98c8e service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:50.246797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 906/1794] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:58:50.307005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:50.309195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:50.314343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:50.324764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9226c0-f190-49b1-995a-44498b748e37 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:50.328506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400d805-b5f2-4db6-bcd7-c09300fd56a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:50.329275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400d805-b5f2-4db6-bcd7-c09300fd56a6 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:50.329730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400d805-b5f2-4db6-bcd7-c09300fd56a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:50.330046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400d805-b5f2-4db6-bcd7-c09300fd56a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:50.331860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1400d805-b5f2-4db6-bcd7-c09300fd56a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:50.358061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04b728-f4f0-4532-92e8-e0e24cb98c8e service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:50.358061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-eb04b728-f4f0-4532-92e8-e0e24cb98c8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:50.425695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400d805-b5f2-4db6-bcd7-c09300fd56a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:50.427566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400d805-b5f2-4db6-bcd7-c09300fd56a6 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:50.464135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 889/1795] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:58:50.481616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04b728-f4f0-4532-92e8-e0e24cb98c8e service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:50.497390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:50.501096 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:50.502748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:50.503057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] PATH_INFO: `/v3/projects/a4d2175740794d35962a282bb41589dc` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:50.510838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d2175740794d35962a282bb41589dc)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:50.510838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:50.730208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d2175740794d35962a282bb41589dc', 'request_id': 'req-9cc334de-bc37-4427-abab-0fd102033a2a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:50.745527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d2175740794d35962a282bb41589dc', 'request_id': 'req-9cc334de-bc37-4427-abab-0fd102033a2a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:50.859977 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d2175740794d35962a282bb41589dc', 'request_id': 'req-9cc334de-bc37-4427-abab-0fd102033a2a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:50.872102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] The token cache is being invalidate because project a4d2175740794d35962a282bb41589dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:50.872486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:50.873138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cc334de-bc37-4427-abab-0fd102033a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:50.874731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 907/1796] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:50 2022] DELETE /identity/v3/projects/a4d2175740794d35962a282bb41589dc => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:50.879804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292ed89-6a91-448a-9339-0b5c32dfa246 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:50.880327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292ed89-6a91-448a-9339-0b5c32dfa246 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:50.880739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292ed89-6a91-448a-9339-0b5c32dfa246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:50.899643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3292ed89-6a91-448a-9339-0b5c32dfa246 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:50.900952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3292ed89-6a91-448a-9339-0b5c32dfa246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:50.967462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 890/1797] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:50.975481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241bc5e-8ceb-4b79-a04a-0b23dee60a29 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:50.975774 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241bc5e-8ceb-4b79-a04a-0b23dee60a29 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:50.976052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241bc5e-8ceb-4b79-a04a-0b23dee60a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:50.997989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-5241bc5e-8ceb-4b79-a04a-0b23dee60a29 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:50.999157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5241bc5e-8ceb-4b79-a04a-0b23dee60a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:51.007265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 908/1798] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:51.011928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292ed89-6a91-448a-9339-0b5c32dfa246 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:51.022740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292ed89-6a91-448a-9339-0b5c32dfa246 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:51.027871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5cfe7-3638-4aac-ade6-152eb0112a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:51.028534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5cfe7-3638-4aac-ade6-152eb0112a03 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:51.028847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5cfe7-3638-4aac-ade6-152eb0112a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:51.029268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5cfe7-3638-4aac-ade6-152eb0112a03 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:51.030070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5cfe7-3638-4aac-ade6-152eb0112a03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:51.030379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5cfe7-3638-4aac-ade6-152eb0112a03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:51.032071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5cfe7-3638-4aac-ade6-152eb0112a03 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:51.046218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 891/1799] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:58:51.050653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241bc5e-8ceb-4b79-a04a-0b23dee60a29 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:51.068221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241bc5e-8ceb-4b79-a04a-0b23dee60a29 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:51.072864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87b405-d5d6-4e9a-b5bc-9ac204656345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:51.074058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87b405-d5d6-4e9a-b5bc-9ac204656345 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:51.074346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87b405-d5d6-4e9a-b5bc-9ac204656345 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:51.074679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87b405-d5d6-4e9a-b5bc-9ac204656345 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:51.075369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87b405-d5d6-4e9a-b5bc-9ac204656345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:51.077274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87b405-d5d6-4e9a-b5bc-9ac204656345 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:51.210506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 909/1800] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:51 2022] POST /identity/v3/projects => generated 363 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:51.214652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5cfe7-3638-4aac-ade6-152eb0112a03 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:51.225732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5cfe7-3638-4aac-ade6-152eb0112a03 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:51.283466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb11fc-26c6-4d28-a9ee-607e8a82fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:51.284139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb11fc-26c6-4d28-a9ee-607e8a82fa6e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:51.284421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb11fc-26c6-4d28-a9ee-607e8a82fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:51.284693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb11fc-26c6-4d28-a9ee-607e8a82fa6e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:51.285382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb11fc-26c6-4d28-a9ee-607e8a82fa6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:51.287332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb11fc-26c6-4d28-a9ee-607e8a82fa6e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:51.413506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 892/1801] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:51 2022] POST /identity/v3/users => generated 282 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:58:51.417794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87b405-d5d6-4e9a-b5bc-9ac204656345 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:51.428498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87b405-d5d6-4e9a-b5bc-9ac204656345 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:51.434259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1088d-3aef-4b2b-a2ef-2dd6cce4a895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:51.434931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1088d-3aef-4b2b-a2ef-2dd6cce4a895 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:51.435211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1088d-3aef-4b2b-a2ef-2dd6cce4a895 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:51.435487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1088d-3aef-4b2b-a2ef-2dd6cce4a895 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:51.436249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1088d-3aef-4b2b-a2ef-2dd6cce4a895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:51.437836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1088d-3aef-4b2b-a2ef-2dd6cce4a895 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:51.449489 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 910/1802] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:51.453616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb11fc-26c6-4d28-a9ee-607e8a82fa6e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:51.463849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb11fc-26c6-4d28-a9ee-607e8a82fa6e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:51.468480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c52559-cb14-4807-8bb7-6a411dee0777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:51.469170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c52559-cb14-4807-8bb7-6a411dee0777 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:51.469457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c52559-cb14-4807-8bb7-6a411dee0777 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:51.469739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c52559-cb14-4807-8bb7-6a411dee0777 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:51.470499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c52559-cb14-4807-8bb7-6a411dee0777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:51.472103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c52559-cb14-4807-8bb7-6a411dee0777 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:51.482350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 893/1803] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:51.486449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1088d-3aef-4b2b-a2ef-2dd6cce4a895 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:51.496668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1088d-3aef-4b2b-a2ef-2dd6cce4a895 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:51.501669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b256-edcd-4619-9fa5-4b4e8aec7ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:51.502583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b256-edcd-4619-9fa5-4b4e8aec7ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:51.502911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b256-edcd-4619-9fa5-4b4e8aec7ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:51.503161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b256-edcd-4619-9fa5-4b4e8aec7ef8 admin admin] PATH_INFO: `/v3/projects/3a5e775d1b2b4f07a469537c2aceb897/users/34cdfd7657564a3a9a9b80cc8954f6cb/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:51.517952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b256-edcd-4619-9fa5-4b4e8aec7ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5e775d1b2b4f07a469537c2aceb897, user_id=34cdfd7657564a3a9a9b80cc8954f6cb, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:51.520638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1b256-edcd-4619-9fa5-4b4e8aec7ef8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:51.528611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:51.625196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 911/1804] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:51 2022] PUT /identity/v3/projects/3a5e775d1b2b4f07a469537c2aceb897/users/34cdfd7657564a3a9a9b80cc8954f6cb/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:51.635271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ccf1b-0519-4444-a2c4-d9dcb4d8234f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:51.635566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ccf1b-0519-4444-a2c4-d9dcb4d8234f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:51.635849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ccf1b-0519-4444-a2c4-d9dcb4d8234f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:51.662102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f72ccf1b-0519-4444-a2c4-d9dcb4d8234f None None] MFA Rules not processed for user `34cdfd7657564a3a9a9b80cc8954f6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:51.663305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f72ccf1b-0519-4444-a2c4-d9dcb4d8234f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:51.726975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 894/1805] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:51.731478 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b256-edcd-4619-9fa5-4b4e8aec7ef8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:51.791339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1b256-edcd-4619-9fa5-4b4e8aec7ef8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:51.796166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af1e7b-a49f-46b0-9ee0-04cd9d3e8456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:51.799320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af1e7b-a49f-46b0-9ee0-04cd9d3e8456 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:51.799320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af1e7b-a49f-46b0-9ee0-04cd9d3e8456 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:51.799320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af1e7b-a49f-46b0-9ee0-04cd9d3e8456 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:51.799320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af1e7b-a49f-46b0-9ee0-04cd9d3e8456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:51.800209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af1e7b-a49f-46b0-9ee0-04cd9d3e8456 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:51.891041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 912/1806] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:51 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:51.892325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ccf1b-0519-4444-a2c4-d9dcb4d8234f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:51.903940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ccf1b-0519-4444-a2c4-d9dcb4d8234f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:51.961763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd181f5-36e3-43a6-b7c5-39b8a6e1a40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:51.962478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd181f5-36e3-43a6-b7c5-39b8a6e1a40f admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:51.962766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd181f5-36e3-43a6-b7c5-39b8a6e1a40f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:51.963051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd181f5-36e3-43a6-b7c5-39b8a6e1a40f admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:51.964441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd181f5-36e3-43a6-b7c5-39b8a6e1a40f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:51.966525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd181f5-36e3-43a6-b7c5-39b8a6e1a40f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:52.080397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 895/1807] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:58:51 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:58:52.084523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af1e7b-a49f-46b0-9ee0-04cd9d3e8456 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:52.094768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af1e7b-a49f-46b0-9ee0-04cd9d3e8456 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:52.099240 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd41b66-9d0d-48fe-9dcf-ae0a1de1cad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:52.099898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41b66-9d0d-48fe-9dcf-ae0a1de1cad9 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:52.100184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41b66-9d0d-48fe-9dcf-ae0a1de1cad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:52.100516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41b66-9d0d-48fe-9dcf-ae0a1de1cad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:52.101194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd41b66-9d0d-48fe-9dcf-ae0a1de1cad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:52.102821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd41b66-9d0d-48fe-9dcf-ae0a1de1cad9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:52.113124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 913/1808] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:52.125687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd181f5-36e3-43a6-b7c5-39b8a6e1a40f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:52.135418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd181f5-36e3-43a6-b7c5-39b8a6e1a40f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:52.140868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fc6a4-37e7-48ba-852e-11bfd96df54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:52.141739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fc6a4-37e7-48ba-852e-11bfd96df54b admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:52.142052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fc6a4-37e7-48ba-852e-11bfd96df54b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:52.142339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7fc6a4-37e7-48ba-852e-11bfd96df54b admin admin] PATH_INFO: `/v3/projects/3e78d686c1314f91bffc411cf0133c66/users/aeae348afa8648c78d1208ef6660225f/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:52.156742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7fc6a4-37e7-48ba-852e-11bfd96df54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e78d686c1314f91bffc411cf0133c66, user_id=aeae348afa8648c78d1208ef6660225f, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:52.159449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7fc6a4-37e7-48ba-852e-11bfd96df54b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:52.252426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 896/1809] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:52 2022] PUT /identity/v3/projects/3e78d686c1314f91bffc411cf0133c66/users/aeae348afa8648c78d1208ef6660225f/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:52.256667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd41b66-9d0d-48fe-9dcf-ae0a1de1cad9 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:52.268679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd41b66-9d0d-48fe-9dcf-ae0a1de1cad9 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:52.352727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe8094-5310-494f-9cc5-3ee12f8f8071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:52.353404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe8094-5310-494f-9cc5-3ee12f8f8071 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:52.354742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe8094-5310-494f-9cc5-3ee12f8f8071 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:52.354742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe8094-5310-494f-9cc5-3ee12f8f8071 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:52.354933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe8094-5310-494f-9cc5-3ee12f8f8071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:52.356580 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe8094-5310-494f-9cc5-3ee12f8f8071 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:52.370351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 914/1810] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:58:52.371824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fc6a4-37e7-48ba-852e-11bfd96df54b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:52.382045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7fc6a4-37e7-48ba-852e-11bfd96df54b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:52.387174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf113bbd-a05c-4999-ad9c-b68b2ea748b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:52.388304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf113bbd-a05c-4999-ad9c-b68b2ea748b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:52.388584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf113bbd-a05c-4999-ad9c-b68b2ea748b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:52.392444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf113bbd-a05c-4999-ad9c-b68b2ea748b1 admin admin] PATH_INFO: `/v3/projects/3e78d686c1314f91bffc411cf0133c66/users/aeae348afa8648c78d1208ef6660225f/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:52.392444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf113bbd-a05c-4999-ad9c-b68b2ea748b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e78d686c1314f91bffc411cf0133c66, user_id=aeae348afa8648c78d1208ef6660225f, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:52.393523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf113bbd-a05c-4999-ad9c-b68b2ea748b1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:52.404272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 897/1811] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:58:52 2022] PUT /identity/v3/projects/3e78d686c1314f91bffc411cf0133c66/users/aeae348afa8648c78d1208ef6660225f/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:52.415541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58489fc-37da-4f65-8fe0-c4847e516824 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:52.415834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58489fc-37da-4f65-8fe0-c4847e516824 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:52.416120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58489fc-37da-4f65-8fe0-c4847e516824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:52.443962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f58489fc-37da-4f65-8fe0-c4847e516824 None None] MFA Rules not processed for user `aeae348afa8648c78d1208ef6660225f`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:52.445175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f58489fc-37da-4f65-8fe0-c4847e516824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:52.515929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 915/1812] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:52.529685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf113bbd-a05c-4999-ad9c-b68b2ea748b1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:52.532484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bf113bbd-a05c-4999-ad9c-b68b2ea748b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:52.600055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf113bbd-a05c-4999-ad9c-b68b2ea748b1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:52.604065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b620305-1512-45ee-a6ab-61acb7a79a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:52.605482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b620305-1512-45ee-a6ab-61acb7a79a09 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:52.606026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b620305-1512-45ee-a6ab-61acb7a79a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:52.608813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b620305-1512-45ee-a6ab-61acb7a79a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:52.617630 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b620305-1512-45ee-a6ab-61acb7a79a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:52.619272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b620305-1512-45ee-a6ab-61acb7a79a09 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:52.631082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 898/1813] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:52 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:52.975143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58489fc-37da-4f65-8fe0-c4847e516824 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:52.985185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58489fc-37da-4f65-8fe0-c4847e516824 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:52.988756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65598001-08b0-482b-97ee-f90eaedbe957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:52.989389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65598001-08b0-482b-97ee-f90eaedbe957 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:52.989668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65598001-08b0-482b-97ee-f90eaedbe957 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:52.989975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65598001-08b0-482b-97ee-f90eaedbe957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:53.048623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65598001-08b0-482b-97ee-f90eaedbe957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:53.050313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65598001-08b0-482b-97ee-f90eaedbe957 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:53.062433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 916/1814] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:52 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:53.733593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:54.763621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:55.516996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:56.782532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b620305-1512-45ee-a6ab-61acb7a79a09 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:56.783918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5b620305-1512-45ee-a6ab-61acb7a79a09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:56.820959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:56.855282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b620305-1512-45ee-a6ab-61acb7a79a09 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:56.859890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:56.860802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:56.861081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:56.861352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] PATH_INFO: `/v3/users/cee47690954c4112a92a4e9d95172bd9` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:56.862569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee47690954c4112a92a4e9d95172bd9)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:56.864495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:57.075954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee47690954c4112a92a4e9d95172bd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:57.076603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee47690954c4112a92a4e9d95172bd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:57.092291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee47690954c4112a92a4e9d95172bd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:57.095440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee47690954c4112a92a4e9d95172bd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:57.206265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee47690954c4112a92a4e9d95172bd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:57.216443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 899/1815] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:56 2022] DELETE /identity/v3/users/cee47690954c4112a92a4e9d95172bd9 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:57.221230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea05d19-15eb-43ee-bba7-8e721cadeaa3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:57.221755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea05d19-15eb-43ee-bba7-8e721cadeaa3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:57.221900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea05d19-15eb-43ee-bba7-8e721cadeaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:57.249400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1ea05d19-15eb-43ee-bba7-8e721cadeaa3 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:57.250667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1ea05d19-15eb-43ee-bba7-8e721cadeaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:57.328993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 917/1816] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:57.338428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:57.341005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:57.396185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:57.437852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b90004-4ee5-4509-b3b5-1baa22312e34 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:57.441481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ad0cf-d237-4344-9b57-f2cecce4410c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:57.442167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ad0cf-d237-4344-9b57-f2cecce4410c service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:57.442448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ad0cf-d237-4344-9b57-f2cecce4410c service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:57.442724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ad0cf-d237-4344-9b57-f2cecce4410c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:57.455065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ad0cf-d237-4344-9b57-f2cecce4410c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:57.456606 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ad0cf-d237-4344-9b57-f2cecce4410c service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:57.467116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 900/1817] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:58.009237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea05d19-15eb-43ee-bba7-8e721cadeaa3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:58.010686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1ea05d19-15eb-43ee-bba7-8e721cadeaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:58.023370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea05d19-15eb-43ee-bba7-8e721cadeaa3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:58.028292 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:58.029364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:58.029663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:58.029971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] PATH_INFO: `/v3/projects/c545774a9d6c427fa49f554b5a3e7b51` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:58.031889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c545774a9d6c427fa49f554b5a3e7b51)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:58.034125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:58.167498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c545774a9d6c427fa49f554b5a3e7b51', 'request_id': 'req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.175813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c545774a9d6c427fa49f554b5a3e7b51', 'request_id': 'req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.362184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c545774a9d6c427fa49f554b5a3e7b51', 'request_id': 'req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.374114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] The token cache is being invalidate because project c545774a9d6c427fa49f554b5a3e7b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:58.374453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.375093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.378403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 918/1818] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:58 2022] DELETE /identity/v3/projects/c545774a9d6c427fa49f554b5a3e7b51 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:58.382803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ad0cf-d237-4344-9b57-f2cecce4410c service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:58.404013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ad0cf-d237-4344-9b57-f2cecce4410c service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:58.467185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:58.468065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:58.468348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:58.468625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] PATH_INFO: `/v3/users/13560b4b5fb94b068bb7f84af8ec57b0` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:58.469989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13560b4b5fb94b068bb7f84af8ec57b0)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:58.471922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:58.663526 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:58.665324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:58.707379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511ea12-d2bd-4b83-a55c-b7fe671d57d7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:58.725861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:58.727198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:58.727660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:58.728086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] PATH_INFO: `/v3/users/6cb057ec20ac454d96170dafe0c68bf0` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:58.730040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cb057ec20ac454d96170dafe0c68bf0)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:58.732362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:58.780369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13560b4b5fb94b068bb7f84af8ec57b0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.781145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13560b4b5fb94b068bb7f84af8ec57b0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.794703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13560b4b5fb94b068bb7f84af8ec57b0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.802494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13560b4b5fb94b068bb7f84af8ec57b0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.922247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2e75bf97-5479-47ef-89a3-e4b1e4db7aa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13560b4b5fb94b068bb7f84af8ec57b0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.928753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 901/1819] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:58 2022] DELETE /identity/v3/users/13560b4b5fb94b068bb7f84af8ec57b0 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:58.973763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cb057ec20ac454d96170dafe0c68bf0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.974455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cb057ec20ac454d96170dafe0c68bf0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.982245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cb057ec20ac454d96170dafe0c68bf0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:58.994607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cb057ec20ac454d96170dafe0c68bf0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:59.025296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:59.127778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cb057ec20ac454d96170dafe0c68bf0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:59.134191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 919/1820] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:58:58 2022] DELETE /identity/v3/users/6cb057ec20ac454d96170dafe0c68bf0 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:59.138934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26854d8-093e-4af8-a79b-09aa7e74497b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:59.139291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26854d8-093e-4af8-a79b-09aa7e74497b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:59.139640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26854d8-093e-4af8-a79b-09aa7e74497b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:59.158329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d26854d8-093e-4af8-a79b-09aa7e74497b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:59.159655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d26854d8-093e-4af8-a79b-09aa7e74497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:59.223274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 902/1821] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:59.232898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:59.234467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:59.313536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8f3c9-6c3c-4ff7-89a0-95dfe7342cd4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:59.317158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808779b-6062-4f94-a8f8-9562ff8dfffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:59.317815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808779b-6062-4f94-a8f8-9562ff8dfffc service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:59.318185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808779b-6062-4f94-a8f8-9562ff8dfffc service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:59.318405 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808779b-6062-4f94-a8f8-9562ff8dfffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:59.328994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808779b-6062-4f94-a8f8-9562ff8dfffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:59.330532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808779b-6062-4f94-a8f8-9562ff8dfffc service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:59.346058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 920/1822] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:58:59.483963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26854d8-093e-4af8-a79b-09aa7e74497b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:59.485499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d26854d8-093e-4af8-a79b-09aa7e74497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:59.509319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26854d8-093e-4af8-a79b-09aa7e74497b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:59.517795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:59.519026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:59.519307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:59.519734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] PATH_INFO: `/v3/projects/92a2eba00ed04b9fb625c3b718c3fafd` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:59.521129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a2eba00ed04b9fb625c3b718c3fafd)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:59.523364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:59.656051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:58:59.679997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a2eba00ed04b9fb625c3b718c3fafd', 'request_id': 'req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:59.688322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a2eba00ed04b9fb625c3b718c3fafd', 'request_id': 'req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:59.763074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a2eba00ed04b9fb625c3b718c3fafd', 'request_id': 'req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:59.779138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] The token cache is being invalidate because project 92a2eba00ed04b9fb625c3b718c3fafd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:58:59.779610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:59.780577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2bec4f25-31c7-40dc-958a-85ed98f8fbc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:58:59.782917 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 903/1823] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:59 2022] DELETE /identity/v3/projects/92a2eba00ed04b9fb625c3b718c3fafd => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:58:59.788119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413912d3-bff4-47b6-bcf4-3b99a3182638 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:59.794133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413912d3-bff4-47b6-bcf4-3b99a3182638 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:59.795358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413912d3-bff4-47b6-bcf4-3b99a3182638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:59.817221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-413912d3-bff4-47b6-bcf4-3b99a3182638 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:59.818481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-413912d3-bff4-47b6-bcf4-3b99a3182638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:59.881367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 921/1824] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:59.891121 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2685aa-6ee4-4a87-b6d5-f5d71c12af06 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:59.891420 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2685aa-6ee4-4a87-b6d5-f5d71c12af06 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:59.891702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2685aa-6ee4-4a87-b6d5-f5d71c12af06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:59.910615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1d2685aa-6ee4-4a87-b6d5-f5d71c12af06 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:58:59.911837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1d2685aa-6ee4-4a87-b6d5-f5d71c12af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:59.920291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 904/1825] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:58:59.924439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413912d3-bff4-47b6-bcf4-3b99a3182638 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:59.935329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413912d3-bff4-47b6-bcf4-3b99a3182638 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:59.942334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5647b68-ad8d-4174-aa71-4d3dce34340f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:59.943247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5647b68-ad8d-4174-aa71-4d3dce34340f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:59.944007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5647b68-ad8d-4174-aa71-4d3dce34340f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:59.944354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5647b68-ad8d-4174-aa71-4d3dce34340f admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:59.946201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5647b68-ad8d-4174-aa71-4d3dce34340f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:58:59.946579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5647b68-ad8d-4174-aa71-4d3dce34340f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:58:59.953011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5647b68-ad8d-4174-aa71-4d3dce34340f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:58:59.958747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2685aa-6ee4-4a87-b6d5-f5d71c12af06 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:59.961331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1d2685aa-6ee4-4a87-b6d5-f5d71c12af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:58:59.976242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 922/1826] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:58:59.984533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5647b68-ad8d-4174-aa71-4d3dce34340f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:58:59.987560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2685aa-6ee4-4a87-b6d5-f5d71c12af06 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:58:59.992849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:58:59.994567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:58:59.995805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:58:59.996230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] PATH_INFO: `/v3/projects/901c8919b73c4680877d340ca513ed68` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:58:59.998101 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901c8919b73c4680877d340ca513ed68)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:00.001418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:00.005457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5647b68-ad8d-4174-aa71-4d3dce34340f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:00.017340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50091838-a066-40b6-b96d-b1eb0996c513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:00.018660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50091838-a066-40b6-b96d-b1eb0996c513 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:00.022229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50091838-a066-40b6-b96d-b1eb0996c513 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:00.022403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50091838-a066-40b6-b96d-b1eb0996c513 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:00.023214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50091838-a066-40b6-b96d-b1eb0996c513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:00.026546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50091838-a066-40b6-b96d-b1eb0996c513 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:00.147002 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 923/1827] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:58:59 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:00.151592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50091838-a066-40b6-b96d-b1eb0996c513 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:00.160758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901c8919b73c4680877d340ca513ed68', 'request_id': 'req-e4ce26b1-93e6-458e-9383-30c57e8e2af0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.166527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901c8919b73c4680877d340ca513ed68', 'request_id': 'req-e4ce26b1-93e6-458e-9383-30c57e8e2af0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.212431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50091838-a066-40b6-b96d-b1eb0996c513 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:00.217722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3007aa-da29-4fe2-8dfc-1e5f5c2224d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:00.218421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3007aa-da29-4fe2-8dfc-1e5f5c2224d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:00.218711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3007aa-da29-4fe2-8dfc-1e5f5c2224d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:00.218992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3007aa-da29-4fe2-8dfc-1e5f5c2224d4 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:00.219785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3007aa-da29-4fe2-8dfc-1e5f5c2224d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:00.221708 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3007aa-da29-4fe2-8dfc-1e5f5c2224d4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:00.253490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901c8919b73c4680877d340ca513ed68', 'request_id': 'req-e4ce26b1-93e6-458e-9383-30c57e8e2af0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.265960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] The token cache is being invalidate because project 901c8919b73c4680877d340ca513ed68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:00.266248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.266895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.268532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 905/1828] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:58:59 2022] DELETE /identity/v3/projects/901c8919b73c4680877d340ca513ed68 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:00.272284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:00.273579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:00.339726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce26b1-93e6-458e-9383-30c57e8e2af0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:00.344314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:00.345188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:00.345468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:00.345742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] PATH_INFO: `/v3/users/6de9d4200d6a46eeb85e58a154b83149` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:00.348384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de9d4200d6a46eeb85e58a154b83149)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:00.348944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:00.365053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 924/1829] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:00 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:00.368931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3007aa-da29-4fe2-8dfc-1e5f5c2224d4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:00.371775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6c3007aa-da29-4fe2-8dfc-1e5f5c2224d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:00.387735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3007aa-da29-4fe2-8dfc-1e5f5c2224d4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:00.392871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93728a-88e0-4b8b-8611-086786b4171f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:00.393549 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93728a-88e0-4b8b-8611-086786b4171f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:00.393837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93728a-88e0-4b8b-8611-086786b4171f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:00.394150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93728a-88e0-4b8b-8611-086786b4171f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:00.394888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93728a-88e0-4b8b-8611-086786b4171f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:00.396548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93728a-88e0-4b8b-8611-086786b4171f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:00.407425 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 925/1830] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:00.411219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93728a-88e0-4b8b-8611-086786b4171f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:00.420966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93728a-88e0-4b8b-8611-086786b4171f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:00.425878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5ffb6-0a9f-4536-95ac-411c4b644161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:00.426618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5ffb6-0a9f-4536-95ac-411c4b644161 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:00.426903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5ffb6-0a9f-4536-95ac-411c4b644161 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:00.427185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5ffb6-0a9f-4536-95ac-411c4b644161 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:00.427907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5ffb6-0a9f-4536-95ac-411c4b644161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:00.429661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5ffb6-0a9f-4536-95ac-411c4b644161 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:00.441729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 926/1831] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:00.445920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5ffb6-0a9f-4536-95ac-411c4b644161 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:00.456474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5ffb6-0a9f-4536-95ac-411c4b644161 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:00.461153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f607cf-497c-46b2-969a-ff47c7672340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:00.462054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f607cf-497c-46b2-969a-ff47c7672340 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:00.462338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f607cf-497c-46b2-969a-ff47c7672340 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:00.462613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f607cf-497c-46b2-969a-ff47c7672340 admin admin] PATH_INFO: `/v3/projects/73c92a51b6654c7cb905c3b042d2ae6f/users/089699ae05a94e6b8bf54f9c3ad87bc7/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:00.476441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f607cf-497c-46b2-969a-ff47c7672340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c92a51b6654c7cb905c3b042d2ae6f, user_id=089699ae05a94e6b8bf54f9c3ad87bc7, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:00.480727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f607cf-497c-46b2-969a-ff47c7672340 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:00.546255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de9d4200d6a46eeb85e58a154b83149'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.546914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de9d4200d6a46eeb85e58a154b83149'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.559084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de9d4200d6a46eeb85e58a154b83149'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.566417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de9d4200d6a46eeb85e58a154b83149'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.569112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 927/1832] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:00 2022] PUT /identity/v3/projects/73c92a51b6654c7cb905c3b042d2ae6f/users/089699ae05a94e6b8bf54f9c3ad87bc7/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:00.579634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c25224-4d8b-4bd6-8986-0d5a7ed8da3c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:00.579947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c25224-4d8b-4bd6-8986-0d5a7ed8da3c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:00.580232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c25224-4d8b-4bd6-8986-0d5a7ed8da3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:00.605760 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-97c25224-4d8b-4bd6-8986-0d5a7ed8da3c None None] MFA Rules not processed for user `089699ae05a94e6b8bf54f9c3ad87bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:00.607000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-97c25224-4d8b-4bd6-8986-0d5a7ed8da3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:00.669528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 928/1833] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:00.673972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c25224-4d8b-4bd6-8986-0d5a7ed8da3c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:00.675307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-97c25224-4d8b-4bd6-8986-0d5a7ed8da3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:00.696308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de9d4200d6a46eeb85e58a154b83149'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:00.703327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 906/1834] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:00 2022] DELETE /identity/v3/users/6de9d4200d6a46eeb85e58a154b83149 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:00.741678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c25224-4d8b-4bd6-8986-0d5a7ed8da3c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:00.799223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e1dd8-f8df-4644-ad23-9baa7a7f9e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:00.800582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e1dd8-f8df-4644-ad23-9baa7a7f9e62 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:00.801054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e1dd8-f8df-4644-ad23-9baa7a7f9e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:00.801369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e1dd8-f8df-4644-ad23-9baa7a7f9e62 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:00.802095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e1dd8-f8df-4644-ad23-9baa7a7f9e62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:00.804059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e1dd8-f8df-4644-ad23-9baa7a7f9e62 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:00.975824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 929/1835] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:00 2022] POST /identity/v3/projects => generated 387 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:00.979952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:01.006466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff8fc-b073-4210-9d6a-4c18bd13cf72 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:01.073768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecadeca-ef8b-4a04-b621-fb989ae1161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:01.074649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecadeca-ef8b-4a04-b621-fb989ae1161e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:01.075612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecadeca-ef8b-4a04-b621-fb989ae1161e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:01.075897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecadeca-ef8b-4a04-b621-fb989ae1161e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:01.076614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecadeca-ef8b-4a04-b621-fb989ae1161e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:01.078704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecadeca-ef8b-4a04-b621-fb989ae1161e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:01.231765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 907/1836] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:00 2022] POST /identity/v3/users => generated 300 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:01.232955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e1dd8-f8df-4644-ad23-9baa7a7f9e62 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:01.243821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e1dd8-f8df-4644-ad23-9baa7a7f9e62 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:01.249169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1000e-2517-4bef-b1a2-9812d279d9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:01.249846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1000e-2517-4bef-b1a2-9812d279d9bb admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:01.250172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1000e-2517-4bef-b1a2-9812d279d9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:01.250486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1000e-2517-4bef-b1a2-9812d279d9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:01.251220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1000e-2517-4bef-b1a2-9812d279d9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:01.252963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1000e-2517-4bef-b1a2-9812d279d9bb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:01.264866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 930/1837] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:01.269205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecadeca-ef8b-4a04-b621-fb989ae1161e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:01.279808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecadeca-ef8b-4a04-b621-fb989ae1161e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:01.285149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3855407-ad4c-451b-b50f-6abf5de54cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:01.286187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3855407-ad4c-451b-b50f-6abf5de54cd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:01.286685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3855407-ad4c-451b-b50f-6abf5de54cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:01.286974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3855407-ad4c-451b-b50f-6abf5de54cd7 admin admin] PATH_INFO: `/v3/projects/ad591394972446b0bdcc496304ee3199/users/211fdd6c5441483baadba87f8c1a6682/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:01.289577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1000e-2517-4bef-b1a2-9812d279d9bb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:01.294047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-edf1000e-2517-4bef-b1a2-9812d279d9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:01.304470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3855407-ad4c-451b-b50f-6abf5de54cd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad591394972446b0bdcc496304ee3199, user_id=211fdd6c5441483baadba87f8c1a6682, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:01.306810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3855407-ad4c-451b-b50f-6abf5de54cd7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:01.313915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1000e-2517-4bef-b1a2-9812d279d9bb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:01.328527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:01.329461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:01.329970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:01.330358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] PATH_INFO: `/v3/projects/0414b5a62af84263956b6d13ab6ff111` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:01.331598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0414b5a62af84263956b6d13ab6ff111)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:01.345003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:01.369229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:01.473257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:01.484056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 908/1838] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:01 2022] PUT /identity/v3/projects/ad591394972446b0bdcc496304ee3199/users/211fdd6c5441483baadba87f8c1a6682/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:01.488029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3855407-ad4c-451b-b50f-6abf5de54cd7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:01.520988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0414b5a62af84263956b6d13ab6ff111', 'request_id': 'req-d4a17da5-377b-465d-ba85-2e4713c30884'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:01.532623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0414b5a62af84263956b6d13ab6ff111', 'request_id': 'req-d4a17da5-377b-465d-ba85-2e4713c30884'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:01.575736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3855407-ad4c-451b-b50f-6abf5de54cd7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:01.655886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0414b5a62af84263956b6d13ab6ff111', 'request_id': 'req-d4a17da5-377b-465d-ba85-2e4713c30884'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:01.664803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1e30d-94a3-49cb-9e0a-0184aea61ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:01.666267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1e30d-94a3-49cb-9e0a-0184aea61ebd admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:01.666723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1e30d-94a3-49cb-9e0a-0184aea61ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:01.667095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1e30d-94a3-49cb-9e0a-0184aea61ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:01.668414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1e30d-94a3-49cb-9e0a-0184aea61ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:01.670583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1e30d-94a3-49cb-9e0a-0184aea61ebd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:01.679181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] The token cache is being invalidate because project 0414b5a62af84263956b6d13ab6ff111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:01.680035 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:01.682224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:01.686677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 931/1839] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:01 2022] DELETE /identity/v3/projects/0414b5a62af84263956b6d13ab6ff111 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:01.686845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 909/1840] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:01.688308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:01.690599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:01.695635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1e30d-94a3-49cb-9e0a-0184aea61ebd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:01.698700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fda1e30d-94a3-49cb-9e0a-0184aea61ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:01.709959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a17da5-377b-465d-ba85-2e4713c30884 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:01.715666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1e30d-94a3-49cb-9e0a-0184aea61ebd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:01.721654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71f8b3-536f-413d-854e-cc22c53646b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:01.722866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71f8b3-536f-413d-854e-cc22c53646b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:01.723327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71f8b3-536f-413d-854e-cc22c53646b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:01.723952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71f8b3-536f-413d-854e-cc22c53646b2 admin admin] PATH_INFO: `/v3/projects/ad591394972446b0bdcc496304ee3199/users/211fdd6c5441483baadba87f8c1a6682/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:01.726339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71f8b3-536f-413d-854e-cc22c53646b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad591394972446b0bdcc496304ee3199, user_id=211fdd6c5441483baadba87f8c1a6682, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:01.729648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:01.730087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71f8b3-536f-413d-854e-cc22c53646b2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:01.730587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:01.730873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:01.731143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] PATH_INFO: `/v3/users/0ecb315931994594a9d61528dc374d95` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:01.732400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ecb315931994594a9d61528dc374d95)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:01.734417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:01.744998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 910/1841] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:01 2022] PUT /identity/v3/projects/ad591394972446b0bdcc496304ee3199/users/211fdd6c5441483baadba87f8c1a6682/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:01.756250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f31a0-ac55-478d-a0fe-d1e1185a0bd3 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:01.757739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f31a0-ac55-478d-a0fe-d1e1185a0bd3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:01.757739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f31a0-ac55-478d-a0fe-d1e1185a0bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:01.784678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6d1f31a0-ac55-478d-a0fe-d1e1185a0bd3 None None] MFA Rules not processed for user `211fdd6c5441483baadba87f8c1a6682`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:01.785916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6d1f31a0-ac55-478d-a0fe-d1e1185a0bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:01.851183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 911/1842] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:01.862588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f31a0-ac55-478d-a0fe-d1e1185a0bd3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:01.864005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6d1f31a0-ac55-478d-a0fe-d1e1185a0bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:01.928402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f31a0-ac55-478d-a0fe-d1e1185a0bd3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:01.933023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810144e-44f9-487d-95a3-894338352db6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:01.933723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810144e-44f9-487d-95a3-894338352db6 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:01.935281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810144e-44f9-487d-95a3-894338352db6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:01.935281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810144e-44f9-487d-95a3-894338352db6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:01.948674 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810144e-44f9-487d-95a3-894338352db6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:01.950856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810144e-44f9-487d-95a3-894338352db6 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:01.966016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 912/1843] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:02.044229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ecb315931994594a9d61528dc374d95'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:02.044992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ecb315931994594a9d61528dc374d95'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:02.052321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ecb315931994594a9d61528dc374d95'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:02.085473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ecb315931994594a9d61528dc374d95'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:02.216889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c73ae716-076f-430d-88a9-9127cacc8cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ecb315931994594a9d61528dc374d95'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:02.223719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 932/1844] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:01 2022] DELETE /identity/v3/users/0ecb315931994594a9d61528dc374d95 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:02.812360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810144e-44f9-487d-95a3-894338352db6 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:02.891561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810144e-44f9-487d-95a3-894338352db6 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:02.958944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:02.959839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:02.960122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:02.960401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] PATH_INFO: `/v3/projects/a675122b4da64becb2f0bfc401f2f8df` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:02.961633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a675122b4da64becb2f0bfc401f2f8df)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:02.963778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:03.084314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a675122b4da64becb2f0bfc401f2f8df', 'request_id': 'req-141b5833-1b5d-4c38-bd69-8575371536a7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:03.089778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a675122b4da64becb2f0bfc401f2f8df', 'request_id': 'req-141b5833-1b5d-4c38-bd69-8575371536a7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:03.176808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a675122b4da64becb2f0bfc401f2f8df', 'request_id': 'req-141b5833-1b5d-4c38-bd69-8575371536a7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:03.189113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] The token cache is being invalidate because project a675122b4da64becb2f0bfc401f2f8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:03.189403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:03.193170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-141b5833-1b5d-4c38-bd69-8575371536a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:03.193170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 913/1845] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:02 2022] DELETE /identity/v3/projects/a675122b4da64becb2f0bfc401f2f8df => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:03.197599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f8c3c-5e0a-4765-8daf-8721a1a2e801 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:03.198094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f8c3c-5e0a-4765-8daf-8721a1a2e801 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:03.198196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f8c3c-5e0a-4765-8daf-8721a1a2e801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:03.221358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1f2f8c3c-5e0a-4765-8daf-8721a1a2e801 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:03.222663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1f2f8c3c-5e0a-4765-8daf-8721a1a2e801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:03.292733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 933/1846] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:03.302180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448e5f0-829b-4fe2-8f9c-e02deb5575ee None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:03.302678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448e5f0-829b-4fe2-8f9c-e02deb5575ee None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:03.303139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448e5f0-829b-4fe2-8f9c-e02deb5575ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:03.331888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4448e5f0-829b-4fe2-8f9c-e02deb5575ee None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:03.333123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4448e5f0-829b-4fe2-8f9c-e02deb5575ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:03.344373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 914/1847] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:03.348680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f8c3c-5e0a-4765-8daf-8721a1a2e801 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:03.360247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f8c3c-5e0a-4765-8daf-8721a1a2e801 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:03.365431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedc9ac-d9df-406e-8c78-a42dd9edce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:03.366175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedc9ac-d9df-406e-8c78-a42dd9edce6d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:03.366462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedc9ac-d9df-406e-8c78-a42dd9edce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:03.366796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedc9ac-d9df-406e-8c78-a42dd9edce6d admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:03.367530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedc9ac-d9df-406e-8c78-a42dd9edce6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:03.367894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedc9ac-d9df-406e-8c78-a42dd9edce6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:03.369506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedc9ac-d9df-406e-8c78-a42dd9edce6d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:03.384804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 934/1848] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:03.389511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448e5f0-829b-4fe2-8f9c-e02deb5575ee None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:03.401839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448e5f0-829b-4fe2-8f9c-e02deb5575ee None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:03.407978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b57190-06bc-4bdc-93bf-82c5b83bdb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:03.408642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b57190-06bc-4bdc-93bf-82c5b83bdb75 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:03.408921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b57190-06bc-4bdc-93bf-82c5b83bdb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:03.409196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b57190-06bc-4bdc-93bf-82c5b83bdb75 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:03.409944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b57190-06bc-4bdc-93bf-82c5b83bdb75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:03.411929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b57190-06bc-4bdc-93bf-82c5b83bdb75 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:03.606660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 915/1849] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:03 2022] POST /identity/v3/projects => generated 389 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:03.617119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedc9ac-d9df-406e-8c78-a42dd9edce6d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:03.629572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedc9ac-d9df-406e-8c78-a42dd9edce6d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:03.719167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be622-dd87-4849-82bb-892a0d0b479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:03.720211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be622-dd87-4849-82bb-892a0d0b479c admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:03.720644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be622-dd87-4849-82bb-892a0d0b479c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:03.721069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92be622-dd87-4849-82bb-892a0d0b479c admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:03.721957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92be622-dd87-4849-82bb-892a0d0b479c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:03.725089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92be622-dd87-4849-82bb-892a0d0b479c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:03.882596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:03.893561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 935/1850] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:03 2022] POST /identity/v3/users => generated 295 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:03.897672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b57190-06bc-4bdc-93bf-82c5b83bdb75 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:03.913099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b57190-06bc-4bdc-93bf-82c5b83bdb75 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:03.920378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60879366-bd04-434f-a3a7-65eaf73ba7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:03.921222 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60879366-bd04-434f-a3a7-65eaf73ba7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:03.921660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60879366-bd04-434f-a3a7-65eaf73ba7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:03.922342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60879366-bd04-434f-a3a7-65eaf73ba7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:03.923623 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60879366-bd04-434f-a3a7-65eaf73ba7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:03.926204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60879366-bd04-434f-a3a7-65eaf73ba7a3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:03.941445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 916/1851] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:03.946027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be622-dd87-4849-82bb-892a0d0b479c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:03.960370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92be622-dd87-4849-82bb-892a0d0b479c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:03.966330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830bbb4a-13e6-4134-acb7-f76f6903e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:03.967015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bbb4a-13e6-4134-acb7-f76f6903e869 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:03.967299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bbb4a-13e6-4134-acb7-f76f6903e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:03.967586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bbb4a-13e6-4134-acb7-f76f6903e869 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:03.968323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830bbb4a-13e6-4134-acb7-f76f6903e869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:03.970001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830bbb4a-13e6-4134-acb7-f76f6903e869 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:03.981635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 936/1852] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:03.989046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60879366-bd04-434f-a3a7-65eaf73ba7a3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:04.001117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60879366-bd04-434f-a3a7-65eaf73ba7a3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:04.006438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4fe61-f890-4222-8bf0-b06587cbfc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:04.007635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4fe61-f890-4222-8bf0-b06587cbfc33 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:04.008071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4fe61-f890-4222-8bf0-b06587cbfc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:04.008346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4fe61-f890-4222-8bf0-b06587cbfc33 admin admin] PATH_INFO: `/v3/projects/6b71d30f4f69401f8a91347d4b1fa5c2/users/e308125602a043b1978f0e798a09c79c/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:04.025273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4fe61-f890-4222-8bf0-b06587cbfc33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b71d30f4f69401f8a91347d4b1fa5c2, user_id=e308125602a043b1978f0e798a09c79c, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:04.027914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4fe61-f890-4222-8bf0-b06587cbfc33 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:04.074195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:04.140021 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 917/1853] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:03 2022] PUT /identity/v3/projects/6b71d30f4f69401f8a91347d4b1fa5c2/users/e308125602a043b1978f0e798a09c79c/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:04.156861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c5234-2bf6-4106-9ab7-a40e22a89f93 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:04.157375 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c5234-2bf6-4106-9ab7-a40e22a89f93 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:04.157656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c5234-2bf6-4106-9ab7-a40e22a89f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:04.160528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4fe61-f890-4222-8bf0-b06587cbfc33 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:04.162372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cab4fe61-f890-4222-8bf0-b06587cbfc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:04.186765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-667c5234-2bf6-4106-9ab7-a40e22a89f93 None None] MFA Rules not processed for user `e308125602a043b1978f0e798a09c79c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:04.188014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-667c5234-2bf6-4106-9ab7-a40e22a89f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:04.226478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4fe61-f890-4222-8bf0-b06587cbfc33 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:04.231163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:04.232050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:04.232335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:04.234151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] PATH_INFO: `/v3/users/089699ae05a94e6b8bf54f9c3ad87bc7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:04.236012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] RBAC: Authorizing `identity:delete_user(user_id=089699ae05a94e6b8bf54f9c3ad87bc7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:04.237992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:04.254301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 937/1854] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:04.258685 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c5234-2bf6-4106-9ab7-a40e22a89f93 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:04.269614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c5234-2bf6-4106-9ab7-a40e22a89f93 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:04.274249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a4983-a986-4312-b2ad-5f7bbfe08042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:04.274894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a4983-a986-4312-b2ad-5f7bbfe08042 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:04.275181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a4983-a986-4312-b2ad-5f7bbfe08042 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:04.275461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a4983-a986-4312-b2ad-5f7bbfe08042 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:04.276692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a4983-a986-4312-b2ad-5f7bbfe08042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:04.278623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a4983-a986-4312-b2ad-5f7bbfe08042 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:04.464487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 938/1855] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:04 2022] POST /identity/v3/projects => generated 389 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:04.468475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a4983-a986-4312-b2ad-5f7bbfe08042 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:04.540298 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a4983-a986-4312-b2ad-5f7bbfe08042 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:04.545546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1959c3e-60dd-4a13-89ac-0aa9203ae952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:04.546252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1959c3e-60dd-4a13-89ac-0aa9203ae952 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:04.546561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1959c3e-60dd-4a13-89ac-0aa9203ae952 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:04.546839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1959c3e-60dd-4a13-89ac-0aa9203ae952 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:04.547559 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1959c3e-60dd-4a13-89ac-0aa9203ae952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:04.549535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1959c3e-60dd-4a13-89ac-0aa9203ae952 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:04.601672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '089699ae05a94e6b8bf54f9c3ad87bc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:04.602390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '089699ae05a94e6b8bf54f9c3ad87bc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:04.617142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '089699ae05a94e6b8bf54f9c3ad87bc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:04.625660 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '089699ae05a94e6b8bf54f9c3ad87bc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:04.748263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 939/1856] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:04 2022] POST /identity/v3/users => generated 301 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:04.749821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1959c3e-60dd-4a13-89ac-0aa9203ae952 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:04.750572 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-46218516-fe14-4a8c-95c2-03c98f84152c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '089699ae05a94e6b8bf54f9c3ad87bc7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:04.751644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b1959c3e-60dd-4a13-89ac-0aa9203ae952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:04.762516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 918/1857] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:04 2022] DELETE /identity/v3/users/089699ae05a94e6b8bf54f9c3ad87bc7 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:04.765525 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1959c3e-60dd-4a13-89ac-0aa9203ae952 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:04.766940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6889449-2222-4506-9a78-ef6c7d1018c0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:04.767333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6889449-2222-4506-9a78-ef6c7d1018c0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:04.767709 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6889449-2222-4506-9a78-ef6c7d1018c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:04.788722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f6889449-2222-4506-9a78-ef6c7d1018c0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:04.790165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f6889449-2222-4506-9a78-ef6c7d1018c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:04.828205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770e180-cdd4-4c2c-ba0c-9d479abcfc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:04.828883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770e180-cdd4-4c2c-ba0c-9d479abcfc6d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:04.829172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770e180-cdd4-4c2c-ba0c-9d479abcfc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:04.829488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770e180-cdd4-4c2c-ba0c-9d479abcfc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:04.830291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770e180-cdd4-4c2c-ba0c-9d479abcfc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:04.831913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770e180-cdd4-4c2c-ba0c-9d479abcfc6d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:04.844107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 940/1858] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:04.849692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770e180-cdd4-4c2c-ba0c-9d479abcfc6d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:04.856557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 919/1859] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:04.861350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770e180-cdd4-4c2c-ba0c-9d479abcfc6d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:04.866066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6889449-2222-4506-9a78-ef6c7d1018c0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:04.866479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f1e10-ed7e-4869-ad87-069935e1c0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:04.867759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f1e10-ed7e-4869-ad87-069935e1c0ed admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:04.868049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f1e10-ed7e-4869-ad87-069935e1c0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:04.868366 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f1e10-ed7e-4869-ad87-069935e1c0ed admin admin] PATH_INFO: `/v3/projects/b28ef8349cf9423890616f94abf1868f/users/79fc8f654b48440895b14823b2f6b89f/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:04.868654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f6889449-2222-4506-9a78-ef6c7d1018c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:04.887453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f1e10-ed7e-4869-ad87-069935e1c0ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28ef8349cf9423890616f94abf1868f, user_id=79fc8f654b48440895b14823b2f6b89f, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:04.890278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f1e10-ed7e-4869-ad87-069935e1c0ed admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:04.931664 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6889449-2222-4506-9a78-ef6c7d1018c0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:04.935325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7693e9c-bc8e-4d4c-baae-2c3f747d2bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:04.935999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7693e9c-bc8e-4d4c-baae-2c3f747d2bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:04.936282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7693e9c-bc8e-4d4c-baae-2c3f747d2bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:04.936559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7693e9c-bc8e-4d4c-baae-2c3f747d2bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:04.945413 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7693e9c-bc8e-4d4c-baae-2c3f747d2bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:04.946870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7693e9c-bc8e-4d4c-baae-2c3f747d2bf1 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:04.956073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 920/1860] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:05.024956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 941/1861] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:04 2022] PUT /identity/v3/projects/b28ef8349cf9423890616f94abf1868f/users/79fc8f654b48440895b14823b2f6b89f/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:05.025420 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7693e9c-bc8e-4d4c-baae-2c3f747d2bf1 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:05.035665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7693e9c-bc8e-4d4c-baae-2c3f747d2bf1 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:05.100074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9354aa3-d9c4-4624-a022-1a00d8e4cf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:05.101004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9354aa3-d9c4-4624-a022-1a00d8e4cf46 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:05.101294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9354aa3-d9c4-4624-a022-1a00d8e4cf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:05.101574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9354aa3-d9c4-4624-a022-1a00d8e4cf46 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:05.102502 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9354aa3-d9c4-4624-a022-1a00d8e4cf46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:05.105068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9354aa3-d9c4-4624-a022-1a00d8e4cf46 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:05.120534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 921/1862] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:05.121778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f1e10-ed7e-4869-ad87-069935e1c0ed admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:05.132551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f1e10-ed7e-4869-ad87-069935e1c0ed admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:05.138585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c9b8b-2b53-4f29-8f15-c05fefab3c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:05.139508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c9b8b-2b53-4f29-8f15-c05fefab3c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:05.139821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c9b8b-2b53-4f29-8f15-c05fefab3c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:05.140038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c9b8b-2b53-4f29-8f15-c05fefab3c07 admin admin] PATH_INFO: `/v3/projects/b28ef8349cf9423890616f94abf1868f/users/79fc8f654b48440895b14823b2f6b89f/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:05.142159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c9b8b-2b53-4f29-8f15-c05fefab3c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28ef8349cf9423890616f94abf1868f, user_id=79fc8f654b48440895b14823b2f6b89f, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:05.145472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c9b8b-2b53-4f29-8f15-c05fefab3c07 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:05.157282 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 942/1863] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:05 2022] PUT /identity/v3/projects/b28ef8349cf9423890616f94abf1868f/users/79fc8f654b48440895b14823b2f6b89f/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:05.168260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f063af-1393-43e9-b176-3c8083e71f0a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:05.168640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f063af-1393-43e9-b176-3c8083e71f0a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:05.168974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f063af-1393-43e9-b176-3c8083e71f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:05.204102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c5f063af-1393-43e9-b176-3c8083e71f0a None None] MFA Rules not processed for user `79fc8f654b48440895b14823b2f6b89f`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:05.205396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c5f063af-1393-43e9-b176-3c8083e71f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:05.283050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 922/1864] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:05.294052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c9b8b-2b53-4f29-8f15-c05fefab3c07 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:05.296781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-998c9b8b-2b53-4f29-8f15-c05fefab3c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:05.378894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c9b8b-2b53-4f29-8f15-c05fefab3c07 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:05.382642 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a88d61-3caa-4e9d-8ed5-a9dd84a43394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:05.383297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a88d61-3caa-4e9d-8ed5-a9dd84a43394 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:05.383587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a88d61-3caa-4e9d-8ed5-a9dd84a43394 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:05.383884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a88d61-3caa-4e9d-8ed5-a9dd84a43394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:05.393825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a88d61-3caa-4e9d-8ed5-a9dd84a43394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:05.395295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a88d61-3caa-4e9d-8ed5-a9dd84a43394 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:05.405665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 943/1865] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:05.537552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f063af-1393-43e9-b176-3c8083e71f0a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:05.599921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f063af-1393-43e9-b176-3c8083e71f0a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:05.605726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:05.606655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:05.606979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:05.607213 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] PATH_INFO: `/v3/users/e308125602a043b1978f0e798a09c79c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:05.608426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e308125602a043b1978f0e798a09c79c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:05.610412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:05.611636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a88d61-3caa-4e9d-8ed5-a9dd84a43394 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:05.613255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e4a88d61-3caa-4e9d-8ed5-a9dd84a43394 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:05.635246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a88d61-3caa-4e9d-8ed5-a9dd84a43394 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:05.642758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:05.644126 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:05.644574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:05.645172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] PATH_INFO: `/v3/projects/73c92a51b6654c7cb905c3b042d2ae6f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:05.647084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c92a51b6654c7cb905c3b042d2ae6f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:05.650190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:05.941029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c92a51b6654c7cb905c3b042d2ae6f', 'request_id': 'req-1039946a-d46d-4141-9cb9-40f6d05cdd4e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:05.948222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c92a51b6654c7cb905c3b042d2ae6f', 'request_id': 'req-1039946a-d46d-4141-9cb9-40f6d05cdd4e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:05.956708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e308125602a043b1978f0e798a09c79c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:05.957477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e308125602a043b1978f0e798a09c79c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:05.975176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e308125602a043b1978f0e798a09c79c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:05.985244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e308125602a043b1978f0e798a09c79c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.113743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c92a51b6654c7cb905c3b042d2ae6f', 'request_id': 'req-1039946a-d46d-4141-9cb9-40f6d05cdd4e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.125215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] The token cache is being invalidate because project 73c92a51b6654c7cb905c3b042d2ae6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:06.125662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.126311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.127877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 944/1866] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:05 2022] DELETE /identity/v3/projects/73c92a51b6654c7cb905c3b042d2ae6f => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:06.131929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:06.133326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:06.235476 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039946a-d46d-4141-9cb9-40f6d05cdd4e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:06.242794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-829b7277-9a56-4389-8a9e-d396fc748b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e308125602a043b1978f0e798a09c79c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.243060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:06.244004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:06.244290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:06.244804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] PATH_INFO: `/v3/users/211fdd6c5441483baadba87f8c1a6682` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:06.246201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=211fdd6c5441483baadba87f8c1a6682)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:06.257486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:06.257644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 923/1867] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:05 2022] DELETE /identity/v3/users/e308125602a043b1978f0e798a09c79c => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:06.271016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e8dd-c69f-476b-b294-4cea74c03abb None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:06.271323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e8dd-c69f-476b-b294-4cea74c03abb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:06.271608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e8dd-c69f-476b-b294-4cea74c03abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:06.315183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ffa8e8dd-c69f-476b-b294-4cea74c03abb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:06.316413 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ffa8e8dd-c69f-476b-b294-4cea74c03abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:06.392680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 924/1868] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:06.403311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8e8dd-c69f-476b-b294-4cea74c03abb None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:06.405186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ffa8e8dd-c69f-476b-b294-4cea74c03abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:06.463942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8e8dd-c69f-476b-b294-4cea74c03abb None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:06.467645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c23b7-18cd-4569-a286-9af2b7a0e67d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:06.468301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c23b7-18cd-4569-a286-9af2b7a0e67d service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:06.468586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c23b7-18cd-4569-a286-9af2b7a0e67d service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:06.468966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c23b7-18cd-4569-a286-9af2b7a0e67d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:06.478321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c23b7-18cd-4569-a286-9af2b7a0e67d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:06.479894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c23b7-18cd-4569-a286-9af2b7a0e67d service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:06.489665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 925/1869] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:06.520158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211fdd6c5441483baadba87f8c1a6682'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.521547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211fdd6c5441483baadba87f8c1a6682'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.533511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211fdd6c5441483baadba87f8c1a6682'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.551970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211fdd6c5441483baadba87f8c1a6682'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.660402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211fdd6c5441483baadba87f8c1a6682'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:06.667353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 945/1870] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:06 2022] DELETE /identity/v3/users/211fdd6c5441483baadba87f8c1a6682 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:07.295943 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c23b7-18cd-4569-a286-9af2b7a0e67d service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:07.309794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c23b7-18cd-4569-a286-9af2b7a0e67d service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:07.347823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:07.349331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:07.376797 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:07.378113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:07.378535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:07.378883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] PATH_INFO: `/v3/projects/ad591394972446b0bdcc496304ee3199` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:07.380362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad591394972446b0bdcc496304ee3199)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:07.382651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:07.438739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e907fa9-d2de-437f-ab97-327661acccc6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:07.446570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:07.446570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:07.446570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:07.446570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] PATH_INFO: `/v3/projects/6b71d30f4f69401f8a91347d4b1fa5c2` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:07.447896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b71d30f4f69401f8a91347d4b1fa5c2)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:07.448324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:07.517205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad591394972446b0bdcc496304ee3199', 'request_id': 'req-d4166948-1148-4fd4-a37f-9291caee52b4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.523693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad591394972446b0bdcc496304ee3199', 'request_id': 'req-d4166948-1148-4fd4-a37f-9291caee52b4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.574270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b71d30f4f69401f8a91347d4b1fa5c2', 'request_id': 'req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.580567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b71d30f4f69401f8a91347d4b1fa5c2', 'request_id': 'req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.628596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad591394972446b0bdcc496304ee3199', 'request_id': 'req-d4166948-1148-4fd4-a37f-9291caee52b4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.645273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] The token cache is being invalidate because project ad591394972446b0bdcc496304ee3199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:07.646388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.647058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4166948-1148-4fd4-a37f-9291caee52b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.652641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 926/1871] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:07 2022] DELETE /identity/v3/projects/ad591394972446b0bdcc496304ee3199 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:07.654067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eab60a-8e06-4b99-9703-91ddc17a8c33 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:07.654374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eab60a-8e06-4b99-9703-91ddc17a8c33 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:07.654648 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eab60a-8e06-4b99-9703-91ddc17a8c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:07.673798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-65eab60a-8e06-4b99-9703-91ddc17a8c33 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:07.675081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-65eab60a-8e06-4b99-9703-91ddc17a8c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:07.721839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b71d30f4f69401f8a91347d4b1fa5c2', 'request_id': 'req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.733740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:07.737804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] The token cache is being invalidate because project 6b71d30f4f69401f8a91347d4b1fa5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:07.742561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.742671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 927/1872] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:07.742773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2909cd89-f0e2-43f3-8b9e-b7f622a1a305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:07.742848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eab60a-8e06-4b99-9703-91ddc17a8c33 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:07.742848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-65eab60a-8e06-4b99-9703-91ddc17a8c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:07.753724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eab60a-8e06-4b99-9703-91ddc17a8c33 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:07.757333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 946/1873] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:07 2022] DELETE /identity/v3/projects/6b71d30f4f69401f8a91347d4b1fa5c2 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:07.760109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:07.763393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:07.763687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:07.763974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] PATH_INFO: `/v3/users/34cdfd7657564a3a9a9b80cc8954f6cb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:07.765333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164280b3-49c6-4b49-a488-b934f2b382f0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:07.765742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164280b3-49c6-4b49-a488-b934f2b382f0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:07.766152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164280b3-49c6-4b49-a488-b934f2b382f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:07.769322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34cdfd7657564a3a9a9b80cc8954f6cb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:07.771301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:07.806247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-164280b3-49c6-4b49-a488-b934f2b382f0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:07.807772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-164280b3-49c6-4b49-a488-b934f2b382f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:07.815988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 947/1874] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:07.817180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164280b3-49c6-4b49-a488-b934f2b382f0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:07.818566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-164280b3-49c6-4b49-a488-b934f2b382f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:07.825632 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:07.831439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164280b3-49c6-4b49-a488-b934f2b382f0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:07.836449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:07.837456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:07.837808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:07.838769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] PATH_INFO: `/v3/users/79fc8f654b48440895b14823b2f6b89f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:07.840176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79fc8f654b48440895b14823b2f6b89f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:07.842335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:08.043544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34cdfd7657564a3a9a9b80cc8954f6cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.044229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34cdfd7657564a3a9a9b80cc8954f6cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.056450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34cdfd7657564a3a9a9b80cc8954f6cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.066224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34cdfd7657564a3a9a9b80cc8954f6cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.077786 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79fc8f654b48440895b14823b2f6b89f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.078546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79fc8f654b48440895b14823b2f6b89f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.085549 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79fc8f654b48440895b14823b2f6b89f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.097783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79fc8f654b48440895b14823b2f6b89f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.146668 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34cdfd7657564a3a9a9b80cc8954f6cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.159286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 928/1875] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:07 2022] DELETE /identity/v3/users/34cdfd7657564a3a9a9b80cc8954f6cb => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:08.160282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:08.161682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:08.245423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35ebdd-0ba1-4519-8ff5-ffb0a670e273 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:08.250715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:08.251941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:08.252282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:08.252573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:08.253366 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:08.253687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:08.254086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b0c9b003-55fd-4477-b467-909ecdc9b833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79fc8f654b48440895b14823b2f6b89f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:08.255537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:08.265687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 948/1876] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:07 2022] DELETE /identity/v3/users/79fc8f654b48440895b14823b2f6b89f => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:08.272922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fb0e8-e318-442a-9cf1-08f9105b6824 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:08.273475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fb0e8-e318-442a-9cf1-08f9105b6824 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:08.274059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fb0e8-e318-442a-9cf1-08f9105b6824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:08.275196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 929/1877] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:08.280420 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:08.282544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:08.300744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1c8fb0e8-e318-442a-9cf1-08f9105b6824 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:08.302838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1c8fb0e8-e318-442a-9cf1-08f9105b6824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:08.303610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb070c-d47e-4af8-a47a-ddc247df672b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:08.433869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 949/1878] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:08.437041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835acc22-c15a-47a3-a102-898590a2186e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:08.440302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835acc22-c15a-47a3-a102-898590a2186e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:08.440627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835acc22-c15a-47a3-a102-898590a2186e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:08.440912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835acc22-c15a-47a3-a102-898590a2186e admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:08.441626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835acc22-c15a-47a3-a102-898590a2186e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:08.446135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835acc22-c15a-47a3-a102-898590a2186e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:08.460628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fb0e8-e318-442a-9cf1-08f9105b6824 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:08.462089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1c8fb0e8-e318-442a-9cf1-08f9105b6824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:08.527828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fb0e8-e318-442a-9cf1-08f9105b6824 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:08.531554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32e7be-45bb-4332-9cef-860422a97a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:08.532276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32e7be-45bb-4332-9cef-860422a97a49 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:08.532628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32e7be-45bb-4332-9cef-860422a97a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:08.532972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32e7be-45bb-4332-9cef-860422a97a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:08.551374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 930/1879] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:08 2022] POST /identity/v3/projects => generated 381 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:08.552682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835acc22-c15a-47a3-a102-898590a2186e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:08.556047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32e7be-45bb-4332-9cef-860422a97a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:08.559731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32e7be-45bb-4332-9cef-860422a97a49 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:08.581783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 950/1880] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:08.628457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835acc22-c15a-47a3-a102-898590a2186e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:08.637932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355199e-9570-415c-bd96-9d8ccfcf1562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:08.637932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355199e-9570-415c-bd96-9d8ccfcf1562 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:08.637932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355199e-9570-415c-bd96-9d8ccfcf1562 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:08.637932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355199e-9570-415c-bd96-9d8ccfcf1562 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:08.638247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355199e-9570-415c-bd96-9d8ccfcf1562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:08.638745 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355199e-9570-415c-bd96-9d8ccfcf1562 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:08.748213 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 931/1881] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:08 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:08.749558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32e7be-45bb-4332-9cef-860422a97a49 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:08.770100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32e7be-45bb-4332-9cef-860422a97a49 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:08.776605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f4e43-5135-4237-b434-d170f257bf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:08.778458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f4e43-5135-4237-b434-d170f257bf4f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:08.779166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f4e43-5135-4237-b434-d170f257bf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:08.779671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f4e43-5135-4237-b434-d170f257bf4f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:08.781132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f4e43-5135-4237-b434-d170f257bf4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:08.783466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f4e43-5135-4237-b434-d170f257bf4f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:08.795163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 951/1882] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:08.799516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355199e-9570-415c-bd96-9d8ccfcf1562 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:08.811123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355199e-9570-415c-bd96-9d8ccfcf1562 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:08.815808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9ffa0-5642-44c6-840b-0e235624a6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:08.816482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9ffa0-5642-44c6-840b-0e235624a6ab admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:08.816773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9ffa0-5642-44c6-840b-0e235624a6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:08.817053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9ffa0-5642-44c6-840b-0e235624a6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:08.817777 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9ffa0-5642-44c6-840b-0e235624a6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:08.819455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9ffa0-5642-44c6-840b-0e235624a6ab admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:08.829669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 932/1883] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:08.833964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f4e43-5135-4237-b434-d170f257bf4f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:08.844404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f4e43-5135-4237-b434-d170f257bf4f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:08.848978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b910009-ef47-4b56-9e3f-3586861ee8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:08.849954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b910009-ef47-4b56-9e3f-3586861ee8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:08.850244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b910009-ef47-4b56-9e3f-3586861ee8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:08.850519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b910009-ef47-4b56-9e3f-3586861ee8fc admin admin] PATH_INFO: `/v3/projects/b7f54ad5ae4f4661b1a365a61c769c87/users/dca58cd6434d4112b2dbdcd57351499e/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:08.865998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b910009-ef47-4b56-9e3f-3586861ee8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f54ad5ae4f4661b1a365a61c769c87, user_id=dca58cd6434d4112b2dbdcd57351499e, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:08.868859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b910009-ef47-4b56-9e3f-3586861ee8fc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:08.886739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9ffa0-5642-44c6-840b-0e235624a6ab admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:08.888814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-afe9ffa0-5642-44c6-840b-0e235624a6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:08.929281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9ffa0-5642-44c6-840b-0e235624a6ab admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:08.938448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:08.939459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:08.939750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:08.940029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] PATH_INFO: `/v3/projects/b28ef8349cf9423890616f94abf1868f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:08.941782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28ef8349cf9423890616f94abf1868f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:08.944166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:08.995076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 952/1884] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:08 2022] PUT /identity/v3/projects/b7f54ad5ae4f4661b1a365a61c769c87/users/dca58cd6434d4112b2dbdcd57351499e/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:09.005512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf58abf-ea7f-4c1c-8ee4-e5c897a2ed41 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:09.005947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf58abf-ea7f-4c1c-8ee4-e5c897a2ed41 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:09.006257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf58abf-ea7f-4c1c-8ee4-e5c897a2ed41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:09.056031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3cf58abf-ea7f-4c1c-8ee4-e5c897a2ed41 None None] MFA Rules not processed for user `dca58cd6434d4112b2dbdcd57351499e`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:09.058027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3cf58abf-ea7f-4c1c-8ee4-e5c897a2ed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:09.103553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28ef8349cf9423890616f94abf1868f', 'request_id': 'req-38807551-db3e-44a9-94f9-a51259d48f9c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.117568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28ef8349cf9423890616f94abf1868f', 'request_id': 'req-38807551-db3e-44a9-94f9-a51259d48f9c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.179415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 953/1885] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:09.190792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf58abf-ea7f-4c1c-8ee4-e5c897a2ed41 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:09.192249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3cf58abf-ea7f-4c1c-8ee4-e5c897a2ed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:09.214007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28ef8349cf9423890616f94abf1868f', 'request_id': 'req-38807551-db3e-44a9-94f9-a51259d48f9c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.227860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] The token cache is being invalidate because project b28ef8349cf9423890616f94abf1868f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:09.228469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.230529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-38807551-db3e-44a9-94f9-a51259d48f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.232847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 933/1886] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:08 2022] DELETE /identity/v3/projects/b28ef8349cf9423890616f94abf1868f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:09.239380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369f6df-6e61-47db-8fc8-9dff609c7291 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:09.240452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369f6df-6e61-47db-8fc8-9dff609c7291 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:09.240915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369f6df-6e61-47db-8fc8-9dff609c7291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:09.261474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4369f6df-6e61-47db-8fc8-9dff609c7291 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:09.261674 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf58abf-ea7f-4c1c-8ee4-e5c897a2ed41 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:09.263005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4369f6df-6e61-47db-8fc8-9dff609c7291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:09.315691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb067b-0889-407c-8419-3bc8cccfe9f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:09.316407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb067b-0889-407c-8419-3bc8cccfe9f0 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:09.316691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb067b-0889-407c-8419-3bc8cccfe9f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:09.316968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb067b-0889-407c-8419-3bc8cccfe9f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:09.319054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-82cb067b-0889-407c-8419-3bc8cccfe9f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:09.328310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 934/1887] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:09.336250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41ef1-c6a3-4ffe-9b5c-995b240c59ff None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:09.336544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41ef1-c6a3-4ffe-9b5c-995b240c59ff None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:09.337184 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41ef1-c6a3-4ffe-9b5c-995b240c59ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:09.358455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8bb41ef1-c6a3-4ffe-9b5c-995b240c59ff None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:09.359767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8bb41ef1-c6a3-4ffe-9b5c-995b240c59ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:09.368228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 935/1888] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:09.372850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41ef1-c6a3-4ffe-9b5c-995b240c59ff None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:09.382247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41ef1-c6a3-4ffe-9b5c-995b240c59ff None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:09.385259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb067b-0889-407c-8419-3bc8cccfe9f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:09.386755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb067b-0889-407c-8419-3bc8cccfe9f0 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:09.388124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:09.388910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:09.389264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:09.389614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:09.390464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:09.390841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:09.392640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:09.397336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 954/1889] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:09.400093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:09.410155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 936/1890] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:59:09.411248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb067b-0889-407c-8419-3bc8cccfe9f0 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:09.441375 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb067b-0889-407c-8419-3bc8cccfe9f0 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:09.457768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c17e7-e892-491f-86f2-ede375fe8888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:09.458296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c17e7-e892-491f-86f2-ede375fe8888 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:09.458583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c17e7-e892-491f-86f2-ede375fe8888 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:09.462860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c17e7-e892-491f-86f2-ede375fe8888 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:09.467200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c17e7-e892-491f-86f2-ede375fe8888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:09.469362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c17e7-e892-491f-86f2-ede375fe8888 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:09.556177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:09.558569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:09.571440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20d834-d932-4716-a0f7-5247fa8bd3bf admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:09.578091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:09.578897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:09.579182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:09.579462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] PATH_INFO: `/v3/projects/3a5e775d1b2b4f07a469537c2aceb897` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:09.580684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5e775d1b2b4f07a469537c2aceb897)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:09.582820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:09.654092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 955/1891] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:09 2022] POST /identity/v3/projects => generated 409 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:09.658115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c17e7-e892-491f-86f2-ede375fe8888 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:09.732081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c17e7-e892-491f-86f2-ede375fe8888 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:09.737359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be881a43-d43f-4888-9c3c-76a675b5a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:09.738070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be881a43-d43f-4888-9c3c-76a675b5a727 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:09.738359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be881a43-d43f-4888-9c3c-76a675b5a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:09.738637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be881a43-d43f-4888-9c3c-76a675b5a727 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:09.739338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be881a43-d43f-4888-9c3c-76a675b5a727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:09.741432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be881a43-d43f-4888-9c3c-76a675b5a727 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:09.783466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5e775d1b2b4f07a469537c2aceb897', 'request_id': 'req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.789683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5e775d1b2b4f07a469537c2aceb897', 'request_id': 'req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.871893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5e775d1b2b4f07a469537c2aceb897', 'request_id': 'req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.891524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 956/1892] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:09 2022] POST /identity/v3/users => generated 305 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:09.892684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be881a43-d43f-4888-9c3c-76a675b5a727 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:09.905543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] The token cache is being invalidate because project 3a5e775d1b2b4f07a469537c2aceb897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:09.906262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.907410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:09.910070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be881a43-d43f-4888-9c3c-76a675b5a727 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:09.911720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 937/1893] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:09 2022] DELETE /identity/v3/projects/3a5e775d1b2b4f07a469537c2aceb897 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:09.919796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:09.921431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:09.992033 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908507a-976b-4b27-ba87-b3e216073bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:09.992297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908507a-976b-4b27-ba87-b3e216073bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:09.992589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908507a-976b-4b27-ba87-b3e216073bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:09.992925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908507a-976b-4b27-ba87-b3e216073bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:09.993669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908507a-976b-4b27-ba87-b3e216073bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:09.995599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908507a-976b-4b27-ba87-b3e216073bc2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:10.009671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 957/1894] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:10.011558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925edaa-9d33-4fc0-a1a6-0c1dbb4a01f4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:10.011708 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908507a-976b-4b27-ba87-b3e216073bc2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:10.014013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1908507a-976b-4b27-ba87-b3e216073bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:10.020365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:10.021504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:10.022499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:10.022781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] PATH_INFO: `/v3/users/aeae348afa8648c78d1208ef6660225f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:10.030612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeae348afa8648c78d1208ef6660225f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:10.030612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:10.032515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908507a-976b-4b27-ba87-b3e216073bc2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:10.038305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9d179-db78-4ac4-b736-d79482720b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:10.039128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9d179-db78-4ac4-b736-d79482720b6d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:10.039518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9d179-db78-4ac4-b736-d79482720b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:10.039929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9d179-db78-4ac4-b736-d79482720b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:10.040771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9d179-db78-4ac4-b736-d79482720b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:10.043553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9d179-db78-4ac4-b736-d79482720b6d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:10.057689 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 958/1895] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:10.061761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9d179-db78-4ac4-b736-d79482720b6d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:10.074329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9d179-db78-4ac4-b736-d79482720b6d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:10.079121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5aa30-3436-42a6-86ec-459d37ddb60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:10.080174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5aa30-3436-42a6-86ec-459d37ddb60f admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:10.080585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5aa30-3436-42a6-86ec-459d37ddb60f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:10.080932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5aa30-3436-42a6-86ec-459d37ddb60f admin admin] PATH_INFO: `/v3/projects/25f510accfb348c09e5310ad8077905a/users/49fadeeabfec4aadaa4acf1a6d02e241/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:10.106578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5aa30-3436-42a6-86ec-459d37ddb60f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f510accfb348c09e5310ad8077905a, user_id=49fadeeabfec4aadaa4acf1a6d02e241, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:10.110088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5aa30-3436-42a6-86ec-459d37ddb60f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:10.202482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 959/1896] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:10 2022] PUT /identity/v3/projects/25f510accfb348c09e5310ad8077905a/users/49fadeeabfec4aadaa4acf1a6d02e241/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:10.212305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5e91a-5f1d-4864-9f3a-e723d725106e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:10.212615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5e91a-5f1d-4864-9f3a-e723d725106e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:10.212909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5e91a-5f1d-4864-9f3a-e723d725106e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:10.269630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b8c5e91a-5f1d-4864-9f3a-e723d725106e None None] MFA Rules not processed for user `49fadeeabfec4aadaa4acf1a6d02e241`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:10.271143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b8c5e91a-5f1d-4864-9f3a-e723d725106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:10.306074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeae348afa8648c78d1208ef6660225f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:10.308272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeae348afa8648c78d1208ef6660225f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:10.350961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:10.360759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 960/1897] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:10.361765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5e91a-5f1d-4864-9f3a-e723d725106e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:10.363277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b8c5e91a-5f1d-4864-9f3a-e723d725106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:10.366864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeae348afa8648c78d1208ef6660225f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:10.399175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeae348afa8648c78d1208ef6660225f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:10.442206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5e91a-5f1d-4864-9f3a-e723d725106e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:10.456351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80405-30c6-4153-b478-69862c9d3461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:10.457048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80405-30c6-4153-b478-69862c9d3461 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:10.457463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80405-30c6-4153-b478-69862c9d3461 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:10.457947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80405-30c6-4153-b478-69862c9d3461 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:10.459053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc80405-30c6-4153-b478-69862c9d3461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:10.460885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc80405-30c6-4153-b478-69862c9d3461 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:10.552133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeae348afa8648c78d1208ef6660225f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:10.566945 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 938/1898] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:09 2022] DELETE /identity/v3/users/aeae348afa8648c78d1208ef6660225f => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:10.618191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 961/1899] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:10 2022] POST /identity/v3/projects => generated 407 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:10.622350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:10.743654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bdbc8-dadb-44f8-80d2-ed813c06e2b6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:10.836597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a145f7-f718-47f2-8f97-324cd0bde56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:10.837430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a145f7-f718-47f2-8f97-324cd0bde56e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:10.837767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a145f7-f718-47f2-8f97-324cd0bde56e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:10.838609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a145f7-f718-47f2-8f97-324cd0bde56e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:10.839325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a145f7-f718-47f2-8f97-324cd0bde56e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:10.841627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a145f7-f718-47f2-8f97-324cd0bde56e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:10.949789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 939/1900] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:10 2022] POST /identity/v3/users => generated 310 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:10.955911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80405-30c6-4153-b478-69862c9d3461 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:10.972952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80405-30c6-4153-b478-69862c9d3461 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:10.986040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb055623-befc-494f-9f03-c0758cf435eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:10.986742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb055623-befc-494f-9f03-c0758cf435eb admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:10.987031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb055623-befc-494f-9f03-c0758cf435eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:10.987311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb055623-befc-494f-9f03-c0758cf435eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:10.988054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb055623-befc-494f-9f03-c0758cf435eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:10.990092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb055623-befc-494f-9f03-c0758cf435eb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:11.012970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 962/1901] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:11.013332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a145f7-f718-47f2-8f97-324cd0bde56e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:11.029838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a145f7-f718-47f2-8f97-324cd0bde56e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:11.035297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35072565-538e-468e-a16d-84bc4d8e63d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:11.036280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35072565-538e-468e-a16d-84bc4d8e63d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.036751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35072565-538e-468e-a16d-84bc4d8e63d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.037188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35072565-538e-468e-a16d-84bc4d8e63d4 admin admin] PATH_INFO: `/v3/projects/7a37c4c51d3e4db288867fd374d95d5b/users/bfe0f3bf5eaa4d5d959efcb2d967ccf7/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.055502 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35072565-538e-468e-a16d-84bc4d8e63d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a37c4c51d3e4db288867fd374d95d5b, user_id=bfe0f3bf5eaa4d5d959efcb2d967ccf7, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:11.058993 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35072565-538e-468e-a16d-84bc4d8e63d4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:11.174422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb055623-befc-494f-9f03-c0758cf435eb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:11.180514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 940/1902] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:11 2022] PUT /identity/v3/projects/7a37c4c51d3e4db288867fd374d95d5b/users/bfe0f3bf5eaa4d5d959efcb2d967ccf7/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:11.189447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb055623-befc-494f-9f03-c0758cf435eb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:11.268422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34b4e8-e2b9-4dc8-8686-12d7a1e5d283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:11.269114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34b4e8-e2b9-4dc8-8686-12d7a1e5d283 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.269402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34b4e8-e2b9-4dc8-8686-12d7a1e5d283 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.269680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34b4e8-e2b9-4dc8-8686-12d7a1e5d283 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.270752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34b4e8-e2b9-4dc8-8686-12d7a1e5d283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:11.272504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34b4e8-e2b9-4dc8-8686-12d7a1e5d283 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:11.285916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 963/1903] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:11.288406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35072565-538e-468e-a16d-84bc4d8e63d4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:11.299927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35072565-538e-468e-a16d-84bc4d8e63d4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:11.305054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dac3f-fd3b-4375-88ee-327b2d46f6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:11.305799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dac3f-fd3b-4375-88ee-327b2d46f6dd admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.306189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dac3f-fd3b-4375-88ee-327b2d46f6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.306325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dac3f-fd3b-4375-88ee-327b2d46f6dd admin admin] PATH_INFO: `/v3/projects/7a37c4c51d3e4db288867fd374d95d5b/users/bfe0f3bf5eaa4d5d959efcb2d967ccf7/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.308559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dac3f-fd3b-4375-88ee-327b2d46f6dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a37c4c51d3e4db288867fd374d95d5b, user_id=bfe0f3bf5eaa4d5d959efcb2d967ccf7, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:11.311272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38dac3f-fd3b-4375-88ee-327b2d46f6dd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:11.327597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 941/1904] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:11 2022] PUT /identity/v3/projects/7a37c4c51d3e4db288867fd374d95d5b/users/bfe0f3bf5eaa4d5d959efcb2d967ccf7/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:11.338620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033f31-eaaf-4f19-a423-708169e78bc0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.339281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033f31-eaaf-4f19-a423-708169e78bc0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.339406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033f31-eaaf-4f19-a423-708169e78bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.370733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-97033f31-eaaf-4f19-a423-708169e78bc0 None None] MFA Rules not processed for user `bfe0f3bf5eaa4d5d959efcb2d967ccf7`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:11.372255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-97033f31-eaaf-4f19-a423-708169e78bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:11.393834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dac3f-fd3b-4375-88ee-327b2d46f6dd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:11.395723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d38dac3f-fd3b-4375-88ee-327b2d46f6dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:11.482776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 964/1905] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:11 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:11.495671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033f31-eaaf-4f19-a423-708169e78bc0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:11.497938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-97033f31-eaaf-4f19-a423-708169e78bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:11.558620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dac3f-fd3b-4375-88ee-327b2d46f6dd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:11.564412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:11.565467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.567157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.567157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] PATH_INFO: `/v3/projects/3e78d686c1314f91bffc411cf0133c66` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.567381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033f31-eaaf-4f19-a423-708169e78bc0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:11.567476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e78d686c1314f91bffc411cf0133c66)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:11.569787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:11.571156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc913e4-acc9-4280-b6b3-76185551c7a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:11.572106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc913e4-acc9-4280-b6b3-76185551c7a5 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.572517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc913e4-acc9-4280-b6b3-76185551c7a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.572947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc913e4-acc9-4280-b6b3-76185551c7a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.585367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc913e4-acc9-4280-b6b3-76185551c7a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:11.586909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc913e4-acc9-4280-b6b3-76185551c7a5 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:11.611393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 965/1906] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:11 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:11.677480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2c2b0-2c3c-4adf-826a-b3db9ac09741 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.677771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2c2b0-2c3c-4adf-826a-b3db9ac09741 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.678119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2c2b0-2c3c-4adf-826a-b3db9ac09741 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.680030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 966/1907] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 09:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 09:59:11.685098 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3140e-f52f-4140-ae6e-2e470d9b7064 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.685620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3140e-f52f-4140-ae6e-2e470d9b7064 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.685939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3140e-f52f-4140-ae6e-2e470d9b7064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.712199 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1cc3140e-f52f-4140-ae6e-2e470d9b7064 None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:11.713473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1cc3140e-f52f-4140-ae6e-2e470d9b7064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:11.746396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e78d686c1314f91bffc411cf0133c66', 'request_id': 'req-7bae09a2-66c2-41dc-a29f-17c079750224'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:11.755659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e78d686c1314f91bffc411cf0133c66', 'request_id': 'req-7bae09a2-66c2-41dc-a29f-17c079750224'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:11.802635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 967/1908] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 09:59:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:11.815943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3140e-f52f-4140-ae6e-2e470d9b7064 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:11.818312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1cc3140e-f52f-4140-ae6e-2e470d9b7064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:11.831572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3140e-f52f-4140-ae6e-2e470d9b7064 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:11.845726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e78d686c1314f91bffc411cf0133c66', 'request_id': 'req-7bae09a2-66c2-41dc-a29f-17c079750224'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:11.858825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] The token cache is being invalidate because project 3e78d686c1314f91bffc411cf0133c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:11.859049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:11.859563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7bae09a2-66c2-41dc-a29f-17c079750224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:11.861385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 942/1909] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:11 2022] DELETE /identity/v3/projects/3e78d686c1314f91bffc411cf0133c66 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:11.867405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aab85a-3115-4ec0-832b-312e4ddf9550 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.867405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aab85a-3115-4ec0-832b-312e4ddf9550 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.867405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aab85a-3115-4ec0-832b-312e4ddf9550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.888297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-42aab85a-3115-4ec0-832b-312e4ddf9550 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:11.889742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-42aab85a-3115-4ec0-832b-312e4ddf9550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:11.903163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24f62f-f1bf-4edd-9664-bad414e017e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:11.903837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24f62f-f1bf-4edd-9664-bad414e017e5 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.904128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24f62f-f1bf-4edd-9664-bad414e017e5 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.906519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24f62f-f1bf-4edd-9664-bad414e017e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.912720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cc24f62f-f1bf-4edd-9664-bad414e017e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:11.938717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24f62f-f1bf-4edd-9664-bad414e017e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:11.938717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24f62f-f1bf-4edd-9664-bad414e017e5 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:11.952243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 968/1910] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:11.965925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 943/1911] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:11.975278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3c322-0107-4d84-ab8e-d118d2ff59f8 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:11.975727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3c322-0107-4d84-ab8e-d118d2ff59f8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:11.975998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3c322-0107-4d84-ab8e-d118d2ff59f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:11.997125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3ae3c322-0107-4d84-ab8e-d118d2ff59f8 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:11.998590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3ae3c322-0107-4d84-ab8e-d118d2ff59f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:12.006703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 969/1912] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:12.011623 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aab85a-3115-4ec0-832b-312e4ddf9550 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:12.019345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:12.022088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aab85a-3115-4ec0-832b-312e4ddf9550 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:12.027689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:12.028363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:12.028771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:12.029057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:12.029784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:12.030144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:12.031796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:12.043754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 944/1913] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:12.048443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3c322-0107-4d84-ab8e-d118d2ff59f8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:12.066131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3c322-0107-4d84-ab8e-d118d2ff59f8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:12.073605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54eeae-6254-4914-a60b-0eb53f9a92d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:12.073605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54eeae-6254-4914-a60b-0eb53f9a92d8 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:12.073605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54eeae-6254-4914-a60b-0eb53f9a92d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:12.074011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54eeae-6254-4914-a60b-0eb53f9a92d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:12.074479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54eeae-6254-4914-a60b-0eb53f9a92d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:12.076789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54eeae-6254-4914-a60b-0eb53f9a92d8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:12.183140 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:12.184897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:12.220074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 970/1914] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:12 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:12.224258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54eeae-6254-4914-a60b-0eb53f9a92d8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:12.329869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c6cef-7294-4e5b-b985-a3e70769f815 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:12.347019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54eeae-6254-4914-a60b-0eb53f9a92d8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:12.347019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644bbad-8725-4c9c-9557-823b01f21759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:12.347019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644bbad-8725-4c9c-9557-823b01f21759 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:12.347019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644bbad-8725-4c9c-9557-823b01f21759 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:12.347394 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644bbad-8725-4c9c-9557-823b01f21759 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:12.347394 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644bbad-8725-4c9c-9557-823b01f21759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:12.348455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644bbad-8725-4c9c-9557-823b01f21759 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:12.382499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bb926-0346-4d04-86a5-7bf0092f2ee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:12.383155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bb926-0346-4d04-86a5-7bf0092f2ee1 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:12.383436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bb926-0346-4d04-86a5-7bf0092f2ee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:12.383708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bb926-0346-4d04-86a5-7bf0092f2ee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:12.386145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-856bb926-0346-4d04-86a5-7bf0092f2ee1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:12.447278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bb926-0346-4d04-86a5-7bf0092f2ee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:12.449760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bb926-0346-4d04-86a5-7bf0092f2ee1 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:12.459066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 945/1915] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:12.522524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 971/1916] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:12 2022] POST /identity/v3/users => generated 284 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:12.545860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bb926-0346-4d04-86a5-7bf0092f2ee1 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:12.554767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bb926-0346-4d04-86a5-7bf0092f2ee1 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:12.560894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd78c50-907b-4cbf-9b2e-e6f570cba13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:12.561604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd78c50-907b-4cbf-9b2e-e6f570cba13b admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:12.561944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd78c50-907b-4cbf-9b2e-e6f570cba13b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:12.562218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd78c50-907b-4cbf-9b2e-e6f570cba13b admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:12.563174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd78c50-907b-4cbf-9b2e-e6f570cba13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:12.564875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd78c50-907b-4cbf-9b2e-e6f570cba13b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:12.576869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 946/1917] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:12.580836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644bbad-8725-4c9c-9557-823b01f21759 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:12.591885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644bbad-8725-4c9c-9557-823b01f21759 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:12.597406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7f879-3c11-40dd-929f-98b3c4f99e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:12.598147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7f879-3c11-40dd-929f-98b3c4f99e4a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:12.598147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7f879-3c11-40dd-929f-98b3c4f99e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:12.598147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7f879-3c11-40dd-929f-98b3c4f99e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:12.598882 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7f879-3c11-40dd-929f-98b3c4f99e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:12.600605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7f879-3c11-40dd-929f-98b3c4f99e4a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:12.610858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 972/1918] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:12.614999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd78c50-907b-4cbf-9b2e-e6f570cba13b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:12.626306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd78c50-907b-4cbf-9b2e-e6f570cba13b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:12.631222 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75a1f3-9677-4e9f-b41f-c4f72bd4ed08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:12.632125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a1f3-9677-4e9f-b41f-c4f72bd4ed08 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:12.632419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a1f3-9677-4e9f-b41f-c4f72bd4ed08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:12.632698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75a1f3-9677-4e9f-b41f-c4f72bd4ed08 admin admin] PATH_INFO: `/v3/projects/743f86f82f814ad79420eb6128028139/users/61d6037ee38542fca3b1232a0dc2e788/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:12.648537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75a1f3-9677-4e9f-b41f-c4f72bd4ed08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=743f86f82f814ad79420eb6128028139, user_id=61d6037ee38542fca3b1232a0dc2e788, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:12.651877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75a1f3-9677-4e9f-b41f-c4f72bd4ed08 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:12.743667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 947/1919] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:12 2022] PUT /identity/v3/projects/743f86f82f814ad79420eb6128028139/users/61d6037ee38542fca3b1232a0dc2e788/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:12.754065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edd5c2-4d37-4bf2-984f-c33338ca6b0c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:12.754415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edd5c2-4d37-4bf2-984f-c33338ca6b0c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:12.754751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edd5c2-4d37-4bf2-984f-c33338ca6b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:12.783327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-16edd5c2-4d37-4bf2-984f-c33338ca6b0c None None] MFA Rules not processed for user `61d6037ee38542fca3b1232a0dc2e788`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:12.785607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-16edd5c2-4d37-4bf2-984f-c33338ca6b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:12.851973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 973/1920] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:12.856445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75a1f3-9677-4e9f-b41f-c4f72bd4ed08 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:12.918631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75a1f3-9677-4e9f-b41f-c4f72bd4ed08 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:12.923339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612f005-5479-485f-b3b0-170ad61b5912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:12.923999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612f005-5479-485f-b3b0-170ad61b5912 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:12.924287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612f005-5479-485f-b3b0-170ad61b5912 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:12.924594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612f005-5479-485f-b3b0-170ad61b5912 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:12.925294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612f005-5479-485f-b3b0-170ad61b5912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:12.927314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612f005-5479-485f-b3b0-170ad61b5912 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:13.052500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 948/1921] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:12 2022] POST /identity/v3/projects => generated 369 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:13.056579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edd5c2-4d37-4bf2-984f-c33338ca6b0c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:13.069358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edd5c2-4d37-4bf2-984f-c33338ca6b0c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:13.130772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb44de0-fe9d-4886-b7fe-b2c03b3201e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:13.131457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb44de0-fe9d-4886-b7fe-b2c03b3201e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:13.131740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb44de0-fe9d-4886-b7fe-b2c03b3201e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:13.132015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb44de0-fe9d-4886-b7fe-b2c03b3201e6 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:13.132716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb44de0-fe9d-4886-b7fe-b2c03b3201e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:13.134677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb44de0-fe9d-4886-b7fe-b2c03b3201e6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:13.237610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 974/1922] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:13 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:13.239012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612f005-5479-485f-b3b0-170ad61b5912 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:13.250667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612f005-5479-485f-b3b0-170ad61b5912 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:13.255347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a971163-e312-4857-9b7c-0c83a8a1f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:13.256016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a971163-e312-4857-9b7c-0c83a8a1f877 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:13.256304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a971163-e312-4857-9b7c-0c83a8a1f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:13.256586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a971163-e312-4857-9b7c-0c83a8a1f877 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:13.257313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a971163-e312-4857-9b7c-0c83a8a1f877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:13.259183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a971163-e312-4857-9b7c-0c83a8a1f877 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:13.270854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 949/1923] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:13.275062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb44de0-fe9d-4886-b7fe-b2c03b3201e6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:13.285243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb44de0-fe9d-4886-b7fe-b2c03b3201e6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:13.290007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a25ad0-00b0-43e9-84e0-1b7a47f1a0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:13.290879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a25ad0-00b0-43e9-84e0-1b7a47f1a0bc admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:13.291176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a25ad0-00b0-43e9-84e0-1b7a47f1a0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:13.291479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a25ad0-00b0-43e9-84e0-1b7a47f1a0bc admin admin] PATH_INFO: `/v3/projects/bbab53fa4d574851854c58f14f18f74d/users/5e07452cd3eb456d8eb5025bf717c38b/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:13.305936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a25ad0-00b0-43e9-84e0-1b7a47f1a0bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbab53fa4d574851854c58f14f18f74d, user_id=5e07452cd3eb456d8eb5025bf717c38b, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:13.309080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a25ad0-00b0-43e9-84e0-1b7a47f1a0bc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:13.354486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:13.459109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 975/1924] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:13 2022] PUT /identity/v3/projects/bbab53fa4d574851854c58f14f18f74d/users/5e07452cd3eb456d8eb5025bf717c38b/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:13.463007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a971163-e312-4857-9b7c-0c83a8a1f877 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:13.473239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a971163-e312-4857-9b7c-0c83a8a1f877 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:13.530581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add3894-706e-46dd-a514-3428aa420226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:13.531281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add3894-706e-46dd-a514-3428aa420226 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:13.531590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add3894-706e-46dd-a514-3428aa420226 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:13.531872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add3894-706e-46dd-a514-3428aa420226 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:13.532599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add3894-706e-46dd-a514-3428aa420226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:13.534228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add3894-706e-46dd-a514-3428aa420226 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:13.544439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 950/1925] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:13.551966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a25ad0-00b0-43e9-84e0-1b7a47f1a0bc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:13.563552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a25ad0-00b0-43e9-84e0-1b7a47f1a0bc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:13.568308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fb884-3590-4108-a127-5efbf695c911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:13.569592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fb884-3590-4108-a127-5efbf695c911 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:13.569898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fb884-3590-4108-a127-5efbf695c911 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:13.570185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fb884-3590-4108-a127-5efbf695c911 admin admin] PATH_INFO: `/v3/projects/bbab53fa4d574851854c58f14f18f74d/users/5e07452cd3eb456d8eb5025bf717c38b/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:13.572176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fb884-3590-4108-a127-5efbf695c911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbab53fa4d574851854c58f14f18f74d, user_id=5e07452cd3eb456d8eb5025bf717c38b, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:13.575194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fb884-3590-4108-a127-5efbf695c911 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:13.586156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 976/1926] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:13 2022] PUT /identity/v3/projects/bbab53fa4d574851854c58f14f18f74d/users/5e07452cd3eb456d8eb5025bf717c38b/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:13.599036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8bc60-3410-402d-90e3-f20a953f3672 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:13.599341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8bc60-3410-402d-90e3-f20a953f3672 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:13.599628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8bc60-3410-402d-90e3-f20a953f3672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:13.626196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b0e8bc60-3410-402d-90e3-f20a953f3672 None None] MFA Rules not processed for user `5e07452cd3eb456d8eb5025bf717c38b`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:13.629068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b0e8bc60-3410-402d-90e3-f20a953f3672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:13.694320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 951/1927] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:13 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:13.705772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fb884-3590-4108-a127-5efbf695c911 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:13.709301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-347fb884-3590-4108-a127-5efbf695c911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:13.765534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fb884-3590-4108-a127-5efbf695c911 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:13.769535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dce78b-c4f1-4f73-82fc-e34bda51e6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:13.770226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dce78b-c4f1-4f73-82fc-e34bda51e6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:13.770513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dce78b-c4f1-4f73-82fc-e34bda51e6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:13.770795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dce78b-c4f1-4f73-82fc-e34bda51e6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:13.780783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dce78b-c4f1-4f73-82fc-e34bda51e6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:13.782257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dce78b-c4f1-4f73-82fc-e34bda51e6f4 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:13.791567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 977/1928] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:14.476819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:15.162211 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:15.480466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:15.836607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:16.794257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:16.891788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8bc60-3410-402d-90e3-f20a953f3672 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:16.954551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8bc60-3410-402d-90e3-f20a953f3672 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:16.959477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:16.960457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:16.960734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:16.961007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] PATH_INFO: `/v3/users/61d6037ee38542fca3b1232a0dc2e788` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:16.962273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d6037ee38542fca3b1232a0dc2e788)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:16.964312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:17.163146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d6037ee38542fca3b1232a0dc2e788'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:17.163811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d6037ee38542fca3b1232a0dc2e788'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:17.175681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d6037ee38542fca3b1232a0dc2e788'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:17.182937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d6037ee38542fca3b1232a0dc2e788'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:17.276778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d6037ee38542fca3b1232a0dc2e788'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:17.294135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 952/1929] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:16 2022] DELETE /identity/v3/users/61d6037ee38542fca3b1232a0dc2e788 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:17.294135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:17.294275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8759e39-16a2-4ed9-bab1-0ffbfad0a842 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:17.294275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8759e39-16a2-4ed9-bab1-0ffbfad0a842 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:17.294275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8759e39-16a2-4ed9-bab1-0ffbfad0a842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:17.311731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e8759e39-16a2-4ed9-bab1-0ffbfad0a842 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:17.313296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e8759e39-16a2-4ed9-bab1-0ffbfad0a842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:17.380739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 978/1930] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:17.390951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:17.392551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:17.456013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a322d7f-c957-4a61-a871-9bdf064acbcd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:17.459860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9160f8-661d-41f9-87f2-318f1aad7da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:17.460513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9160f8-661d-41f9-87f2-318f1aad7da5 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:17.460798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9160f8-661d-41f9-87f2-318f1aad7da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:17.461114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9160f8-661d-41f9-87f2-318f1aad7da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:17.472817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9160f8-661d-41f9-87f2-318f1aad7da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:17.475082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9160f8-661d-41f9-87f2-318f1aad7da5 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:17.485184 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 953/1931] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:18.061604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8759e39-16a2-4ed9-bab1-0ffbfad0a842 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:18.063192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e8759e39-16a2-4ed9-bab1-0ffbfad0a842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:18.077867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8759e39-16a2-4ed9-bab1-0ffbfad0a842 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:18.082726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:18.083829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:18.084129 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:18.084445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] PATH_INFO: `/v3/projects/743f86f82f814ad79420eb6128028139` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:18.085815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=743f86f82f814ad79420eb6128028139)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:18.090331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:18.262545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743f86f82f814ad79420eb6128028139', 'request_id': 'req-2ca2dd34-cb63-4997-9f1c-375a7607d241'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.268650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743f86f82f814ad79420eb6128028139', 'request_id': 'req-2ca2dd34-cb63-4997-9f1c-375a7607d241'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.373002 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743f86f82f814ad79420eb6128028139', 'request_id': 'req-2ca2dd34-cb63-4997-9f1c-375a7607d241'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.385585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] The token cache is being invalidate because project 743f86f82f814ad79420eb6128028139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:18.386017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.386738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.388646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 979/1932] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:18 2022] DELETE /identity/v3/projects/743f86f82f814ad79420eb6128028139 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:18.394223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9160f8-661d-41f9-87f2-318f1aad7da5 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:18.405148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9160f8-661d-41f9-87f2-318f1aad7da5 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:18.464058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:18.464956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:18.465238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:18.465535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] PATH_INFO: `/v3/users/5e07452cd3eb456d8eb5025bf717c38b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:18.466821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e07452cd3eb456d8eb5025bf717c38b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:18.468734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:18.496504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:18.686086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e07452cd3eb456d8eb5025bf717c38b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.686740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e07452cd3eb456d8eb5025bf717c38b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.698767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e07452cd3eb456d8eb5025bf717c38b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.705353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e07452cd3eb456d8eb5025bf717c38b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.784954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17089518-89b5-4908-bce0-e204b94b9e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e07452cd3eb456d8eb5025bf717c38b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:18.793341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 954/1933] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:18 2022] DELETE /identity/v3/users/5e07452cd3eb456d8eb5025bf717c38b => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:19.347360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:19.347360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:19.358912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2dd34-cb63-4997-9f1c-375a7607d241 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:19.453160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:19.454255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:19.454816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:19.455414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] PATH_INFO: `/v3/projects/bbab53fa4d574851854c58f14f18f74d` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:19.456821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbab53fa4d574851854c58f14f18f74d)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:19.458984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:19.528261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:19.579134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbab53fa4d574851854c58f14f18f74d', 'request_id': 'req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:19.584385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbab53fa4d574851854c58f14f18f74d', 'request_id': 'req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:19.702326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbab53fa4d574851854c58f14f18f74d', 'request_id': 'req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:19.713609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] The token cache is being invalidate because project bbab53fa4d574851854c58f14f18f74d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:19.713918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:19.714557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-88b637a8-3982-4dce-b3f8-ecf55d13d7bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:19.716220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 980/1934] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:19 2022] DELETE /identity/v3/projects/bbab53fa4d574851854c58f14f18f74d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:19.721110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba60241-39f2-4abf-a46f-2925b46bc29e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:19.721401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba60241-39f2-4abf-a46f-2925b46bc29e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:19.721681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba60241-39f2-4abf-a46f-2925b46bc29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:19.741827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-eba60241-39f2-4abf-a46f-2925b46bc29e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:19.743047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-eba60241-39f2-4abf-a46f-2925b46bc29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:19.805478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 955/1935] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:19.813841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97146e5-eaad-4894-a306-ab1df8ecdf6b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:19.814213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97146e5-eaad-4894-a306-ab1df8ecdf6b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:19.814500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97146e5-eaad-4894-a306-ab1df8ecdf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:19.834181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b97146e5-eaad-4894-a306-ab1df8ecdf6b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:19.835435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b97146e5-eaad-4894-a306-ab1df8ecdf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:19.844128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 981/1936] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:19.848522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba60241-39f2-4abf-a46f-2925b46bc29e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:19.858629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba60241-39f2-4abf-a46f-2925b46bc29e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:19.863369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846463c-ce1a-469f-ba9a-0d1f3b3887ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:19.905410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846463c-ce1a-469f-ba9a-0d1f3b3887ca admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:19.905410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846463c-ce1a-469f-ba9a-0d1f3b3887ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:19.905410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846463c-ce1a-469f-ba9a-0d1f3b3887ca admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:19.905410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846463c-ce1a-469f-ba9a-0d1f3b3887ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:19.905410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846463c-ce1a-469f-ba9a-0d1f3b3887ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:19.905410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846463c-ce1a-469f-ba9a-0d1f3b3887ca admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:19.905910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 956/1937] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:59:19.908657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97146e5-eaad-4894-a306-ab1df8ecdf6b None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:19.918881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97146e5-eaad-4894-a306-ab1df8ecdf6b None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:19.924133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45516ed-6ba6-4fca-aafb-5343644da9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:19.924913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45516ed-6ba6-4fca-aafb-5343644da9ec admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:19.925308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45516ed-6ba6-4fca-aafb-5343644da9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:19.925654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45516ed-6ba6-4fca-aafb-5343644da9ec admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:19.926657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45516ed-6ba6-4fca-aafb-5343644da9ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:19.929144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45516ed-6ba6-4fca-aafb-5343644da9ec admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:20.021201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 982/1938] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:19 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:20.026273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846463c-ce1a-469f-ba9a-0d1f3b3887ca admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:20.036057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846463c-ce1a-469f-ba9a-0d1f3b3887ca admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:20.091520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb641c0f-8330-4565-9ba7-f320da480116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:20.092187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb641c0f-8330-4565-9ba7-f320da480116 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:20.092475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb641c0f-8330-4565-9ba7-f320da480116 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:20.092757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb641c0f-8330-4565-9ba7-f320da480116 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:20.093445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb641c0f-8330-4565-9ba7-f320da480116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:20.095458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb641c0f-8330-4565-9ba7-f320da480116 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:20.247820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 957/1939] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:20 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:20.251738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45516ed-6ba6-4fca-aafb-5343644da9ec admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:20.262665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45516ed-6ba6-4fca-aafb-5343644da9ec admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:20.267241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55dd53-54be-4a5c-b893-1d0b9400c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:20.267901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55dd53-54be-4a5c-b893-1d0b9400c28b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:20.268187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55dd53-54be-4a5c-b893-1d0b9400c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:20.268466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55dd53-54be-4a5c-b893-1d0b9400c28b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:20.269197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55dd53-54be-4a5c-b893-1d0b9400c28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:20.270896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55dd53-54be-4a5c-b893-1d0b9400c28b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:20.281842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 983/1940] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:20.285842 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb641c0f-8330-4565-9ba7-f320da480116 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:20.295798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb641c0f-8330-4565-9ba7-f320da480116 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:20.300446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30036c35-8747-41e0-a54f-61fab1f4bbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:20.301100 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30036c35-8747-41e0-a54f-61fab1f4bbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:20.301387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30036c35-8747-41e0-a54f-61fab1f4bbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:20.301668 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30036c35-8747-41e0-a54f-61fab1f4bbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:20.302458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30036c35-8747-41e0-a54f-61fab1f4bbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:20.304107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30036c35-8747-41e0-a54f-61fab1f4bbc9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:20.314071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 958/1941] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:20.333462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55dd53-54be-4a5c-b893-1d0b9400c28b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:20.343793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55dd53-54be-4a5c-b893-1d0b9400c28b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:20.348257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f75afe-1faf-41b8-80bd-31f528afd59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:20.349167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f75afe-1faf-41b8-80bd-31f528afd59e admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:20.349671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f75afe-1faf-41b8-80bd-31f528afd59e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:20.350141 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f75afe-1faf-41b8-80bd-31f528afd59e admin admin] PATH_INFO: `/v3/projects/4e1793ba5f584cb0a9005c50d80c5c06/users/3dd300436c9545faa56d60ac290bc987/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:20.364489 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f75afe-1faf-41b8-80bd-31f528afd59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1793ba5f584cb0a9005c50d80c5c06, user_id=3dd300436c9545faa56d60ac290bc987, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:20.367181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f75afe-1faf-41b8-80bd-31f528afd59e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:20.384679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:20.459014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 984/1942] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:20 2022] PUT /identity/v3/projects/4e1793ba5f584cb0a9005c50d80c5c06/users/3dd300436c9545faa56d60ac290bc987/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:20.464802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1eaa9-f77f-4b31-b8f5-b0c73cdb2663 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:20.465148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1eaa9-f77f-4b31-b8f5-b0c73cdb2663 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:20.465430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1eaa9-f77f-4b31-b8f5-b0c73cdb2663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:20.484595 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bee1eaa9-f77f-4b31-b8f5-b0c73cdb2663 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:20.485872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bee1eaa9-f77f-4b31-b8f5-b0c73cdb2663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:20.549400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 959/1943] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:20.559630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f75afe-1faf-41b8-80bd-31f528afd59e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:20.561675 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-84f75afe-1faf-41b8-80bd-31f528afd59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:20.620699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f75afe-1faf-41b8-80bd-31f528afd59e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:20.624347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d2e35-e3d8-4aab-bd69-0de5a996e7a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:20.624994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d2e35-e3d8-4aab-bd69-0de5a996e7a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:20.625280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d2e35-e3d8-4aab-bd69-0de5a996e7a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:20.625554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d2e35-e3d8-4aab-bd69-0de5a996e7a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:20.634678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d2e35-e3d8-4aab-bd69-0de5a996e7a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:20.636159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d2e35-e3d8-4aab-bd69-0de5a996e7a2 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:20.646292 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 985/1944] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:20.762885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:21.752303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:21.802226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:22.071873 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58539108-b875-4c6a-8e53-9c852120e270 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:22.072164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58539108-b875-4c6a-8e53-9c852120e270 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:22.072561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58539108-b875-4c6a-8e53-9c852120e270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:22.100353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-58539108-b875-4c6a-8e53-9c852120e270 None None] MFA Rules not processed for user `3dd300436c9545faa56d60ac290bc987`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:22.101530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-58539108-b875-4c6a-8e53-9c852120e270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:22.164437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 960/1945] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:22.176219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d2e35-e3d8-4aab-bd69-0de5a996e7a2 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:22.177675 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7a3d2e35-e3d8-4aab-bd69-0de5a996e7a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:22.235485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d2e35-e3d8-4aab-bd69-0de5a996e7a2 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:22.239094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53dde6-bb30-49ec-9ece-42fa2bbf549e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:22.239740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53dde6-bb30-49ec-9ece-42fa2bbf549e service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:22.240028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53dde6-bb30-49ec-9ece-42fa2bbf549e service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:22.240310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53dde6-bb30-49ec-9ece-42fa2bbf549e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:22.249097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53dde6-bb30-49ec-9ece-42fa2bbf549e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:22.250760 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53dde6-bb30-49ec-9ece-42fa2bbf549e service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:22.260092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 986/1946] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:22.545281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:23.126195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:23.563444 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:23.976882 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:24.363760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340194c5-0cde-4dbb-9c66-1d06978c9107 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:24.364251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340194c5-0cde-4dbb-9c66-1d06978c9107 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:24.364362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340194c5-0cde-4dbb-9c66-1d06978c9107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:24.383815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-340194c5-0cde-4dbb-9c66-1d06978c9107 None None] MFA Rules not processed for user `3dd300436c9545faa56d60ac290bc987`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:24.385657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-340194c5-0cde-4dbb-9c66-1d06978c9107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:24.392927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 961/1947] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:24.587906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53dde6-bb30-49ec-9ece-42fa2bbf549e service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:24.589680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7e53dde6-bb30-49ec-9ece-42fa2bbf549e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:24.604898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53dde6-bb30-49ec-9ece-42fa2bbf549e service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:24.609855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:24.610877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:24.611156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:24.611670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] PATH_INFO: `/v3/users/dca58cd6434d4112b2dbdcd57351499e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:24.613038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca58cd6434d4112b2dbdcd57351499e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:24.615751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:24.756780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:24.845155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca58cd6434d4112b2dbdcd57351499e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:24.845814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca58cd6434d4112b2dbdcd57351499e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:24.853143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca58cd6434d4112b2dbdcd57351499e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:24.866490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca58cd6434d4112b2dbdcd57351499e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:25.179060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca58cd6434d4112b2dbdcd57351499e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:25.187732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 987/1948] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:24 2022] DELETE /identity/v3/users/dca58cd6434d4112b2dbdcd57351499e => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:25.192824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293f690-08dd-4a4c-b023-0b9d0c509a42 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:25.193125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293f690-08dd-4a4c-b023-0b9d0c509a42 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:25.193411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293f690-08dd-4a4c-b023-0b9d0c509a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:25.216987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c293f690-08dd-4a4c-b023-0b9d0c509a42 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:25.218236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c293f690-08dd-4a4c-b023-0b9d0c509a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:25.307545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 962/1949] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:25.318636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:25.320950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:25.383210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b6a1-aa02-4a3a-b22d-8b46562d3c17 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:25.394388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f08376-f5ca-4b1e-ac53-d3757c7ec0b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:25.394388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f08376-f5ca-4b1e-ac53-d3757c7ec0b2 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:25.394388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f08376-f5ca-4b1e-ac53-d3757c7ec0b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:25.394388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f08376-f5ca-4b1e-ac53-d3757c7ec0b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:25.396435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:25.401098 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f08376-f5ca-4b1e-ac53-d3757c7ec0b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:25.402652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f08376-f5ca-4b1e-ac53-d3757c7ec0b2 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:25.413422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 988/1950] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:26.117359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293f690-08dd-4a4c-b023-0b9d0c509a42 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:26.120504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c293f690-08dd-4a4c-b023-0b9d0c509a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:26.134006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f08376-f5ca-4b1e-ac53-d3757c7ec0b2 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:26.136319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f1f08376-f5ca-4b1e-ac53-d3757c7ec0b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:26.147680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293f690-08dd-4a4c-b023-0b9d0c509a42 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:26.151334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f08376-f5ca-4b1e-ac53-d3757c7ec0b2 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:26.152527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:26.153433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:26.153719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:26.154046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] PATH_INFO: `/v3/projects/b7f54ad5ae4f4661b1a365a61c769c87` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:26.155379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f54ad5ae4f4661b1a365a61c769c87)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:26.156180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:26.157042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:26.157309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:26.157427 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:26.157583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] PATH_INFO: `/v3/users/e5c1bbcf86214f8c896aa6d67203f0f9` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:26.158838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c1bbcf86214f8c896aa6d67203f0f9)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:26.160786 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:26.429603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f54ad5ae4f4661b1a365a61c769c87', 'request_id': 'req-5885bf81-cbf5-48a0-a55c-ff8726afd53e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.437006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f54ad5ae4f4661b1a365a61c769c87', 'request_id': 'req-5885bf81-cbf5-48a0-a55c-ff8726afd53e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.441459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c1bbcf86214f8c896aa6d67203f0f9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.442206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c1bbcf86214f8c896aa6d67203f0f9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.451282 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c1bbcf86214f8c896aa6d67203f0f9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.464063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c1bbcf86214f8c896aa6d67203f0f9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.522565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f54ad5ae4f4661b1a365a61c769c87', 'request_id': 'req-5885bf81-cbf5-48a0-a55c-ff8726afd53e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.534837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] The token cache is being invalidate because project b7f54ad5ae4f4661b1a365a61c769c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:26.535131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.535781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5885bf81-cbf5-48a0-a55c-ff8726afd53e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.537653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 963/1951] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:26 2022] DELETE /identity/v3/projects/b7f54ad5ae4f4661b1a365a61c769c87 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:26.543487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad5b2e-23c1-4c82-9813-debac6ca9fa5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:26.543943 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad5b2e-23c1-4c82-9813-debac6ca9fa5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:26.544221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad5b2e-23c1-4c82-9813-debac6ca9fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:26.563786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e7ad5b2e-23c1-4c82-9813-debac6ca9fa5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:26.565022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e7ad5b2e-23c1-4c82-9813-debac6ca9fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:26.626910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 964/1952] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:26.634812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0d296-69a9-4da0-a975-6f2b7d3684a1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:26.635108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0d296-69a9-4da0-a975-6f2b7d3684a1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:26.635379 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0d296-69a9-4da0-a975-6f2b7d3684a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:26.654046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-80c0d296-69a9-4da0-a975-6f2b7d3684a1 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:26.655287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-80c0d296-69a9-4da0-a975-6f2b7d3684a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:26.665793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 965/1953] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:26.670129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0d296-69a9-4da0-a975-6f2b7d3684a1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:26.671301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:26.679296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0d296-69a9-4da0-a975-6f2b7d3684a1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:26.684444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986a232-ebb3-4338-a92e-f86ac9de9c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:26.685090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986a232-ebb3-4338-a92e-f86ac9de9c8c admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:26.687692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986a232-ebb3-4338-a92e-f86ac9de9c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:26.687692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986a232-ebb3-4338-a92e-f86ac9de9c8c admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:26.687692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986a232-ebb3-4338-a92e-f86ac9de9c8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:26.687692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986a232-ebb3-4338-a92e-f86ac9de9c8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:26.688357 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986a232-ebb3-4338-a92e-f86ac9de9c8c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:26.700357 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 966/1954] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:59:26.708602 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986a232-ebb3-4338-a92e-f86ac9de9c8c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:26.726901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986a232-ebb3-4338-a92e-f86ac9de9c8c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:26.726997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-99c6e07c-bb35-409d-9ea3-633ad9d29020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c1bbcf86214f8c896aa6d67203f0f9'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:26.727085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:26.731031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37af9-b91f-4e4c-9ade-1a9729f789d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:26.732174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37af9-b91f-4e4c-9ade-1a9729f789d8 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:26.732634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37af9-b91f-4e4c-9ade-1a9729f789d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:26.733085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37af9-b91f-4e4c-9ade-1a9729f789d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:26.733983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d37af9-b91f-4e4c-9ade-1a9729f789d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:26.736775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 989/1955] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:26 2022] DELETE /identity/v3/users/e5c1bbcf86214f8c896aa6d67203f0f9 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:26.738497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742309f-4373-4201-96f8-96616c0c18ad None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:26.738918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742309f-4373-4201-96f8-96616c0c18ad None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:26.739065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d37af9-b91f-4e4c-9ade-1a9729f789d8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:26.739495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742309f-4373-4201-96f8-96616c0c18ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:26.762507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9742309f-4373-4201-96f8-96616c0c18ad None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:26.763771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9742309f-4373-4201-96f8-96616c0c18ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:26.824557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 967/1956] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:26 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:26.826204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37af9-b91f-4e4c-9ade-1a9729f789d8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:26.833665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 990/1957] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:26.840790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742309f-4373-4201-96f8-96616c0c18ad None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:26.842198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9742309f-4373-4201-96f8-96616c0c18ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:26.907508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37af9-b91f-4e4c-9ade-1a9729f789d8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:26.908441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742309f-4373-4201-96f8-96616c0c18ad None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:26.912411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e7fb-d4ce-406a-92b5-5f14781a0ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:26.913082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e7fb-d4ce-406a-92b5-5f14781a0ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:26.913369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e7fb-d4ce-406a-92b5-5f14781a0ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:26.913647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e7fb-d4ce-406a-92b5-5f14781a0ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:26.914375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4e7fb-d4ce-406a-92b5-5f14781a0ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:26.916330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4e7fb-d4ce-406a-92b5-5f14781a0ab6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:26.956398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1462e-86cb-4da1-be79-9eacf84c734f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:26.957041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1462e-86cb-4da1-be79-9eacf84c734f service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:26.957448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1462e-86cb-4da1-be79-9eacf84c734f service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:26.957600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1462e-86cb-4da1-be79-9eacf84c734f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:26.966712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1462e-86cb-4da1-be79-9eacf84c734f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:26.968138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1462e-86cb-4da1-be79-9eacf84c734f service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:26.977438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 991/1958] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:27.057310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 968/1959] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:26 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:27.058628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1462e-86cb-4da1-be79-9eacf84c734f service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:27.068547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1462e-86cb-4da1-be79-9eacf84c734f service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:27.078553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ded3e3-9bfb-4858-a98b-b72ff297a3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:27.079239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ded3e3-9bfb-4858-a98b-b72ff297a3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.079935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ded3e3-9bfb-4858-a98b-b72ff297a3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.079935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ded3e3-9bfb-4858-a98b-b72ff297a3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.080639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ded3e3-9bfb-4858-a98b-b72ff297a3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:27.082398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ded3e3-9bfb-4858-a98b-b72ff297a3b3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:27.099662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 992/1960] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:27.100906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e7fb-d4ce-406a-92b5-5f14781a0ab6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:27.111369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e7fb-d4ce-406a-92b5-5f14781a0ab6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:27.116824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2da87d-82d2-4614-89b9-8b9e4064fc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:27.117499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2da87d-82d2-4614-89b9-8b9e4064fc4a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.118063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2da87d-82d2-4614-89b9-8b9e4064fc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.118169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2da87d-82d2-4614-89b9-8b9e4064fc4a admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.118834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2da87d-82d2-4614-89b9-8b9e4064fc4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:27.120638 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2da87d-82d2-4614-89b9-8b9e4064fc4a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:27.131065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 969/1961] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:27.135632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ded3e3-9bfb-4858-a98b-b72ff297a3b3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:27.146536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ded3e3-9bfb-4858-a98b-b72ff297a3b3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:27.151814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7657325-8834-4520-9ee6-1649f78555b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:27.152700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7657325-8834-4520-9ee6-1649f78555b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.152986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7657325-8834-4520-9ee6-1649f78555b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.153266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7657325-8834-4520-9ee6-1649f78555b8 admin admin] PATH_INFO: `/v3/projects/d4249c6d236241ec892a304cc57d03e9/users/9c6e4c7f415643429e54f1d1897b6b47/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.169998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7657325-8834-4520-9ee6-1649f78555b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4249c6d236241ec892a304cc57d03e9, user_id=9c6e4c7f415643429e54f1d1897b6b47, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:27.172718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7657325-8834-4520-9ee6-1649f78555b8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:27.189912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2da87d-82d2-4614-89b9-8b9e4064fc4a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:27.197961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0d2da87d-82d2-4614-89b9-8b9e4064fc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:27.208370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2da87d-82d2-4614-89b9-8b9e4064fc4a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:27.214018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:27.215019 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.215180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.215448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] PATH_INFO: `/v3/users/49fadeeabfec4aadaa4acf1a6d02e241` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.216665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49fadeeabfec4aadaa4acf1a6d02e241)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:27.218775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:27.252247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:27.261327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 993/1962] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:27 2022] PUT /identity/v3/projects/d4249c6d236241ec892a304cc57d03e9/users/9c6e4c7f415643429e54f1d1897b6b47/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:27.271793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21beb74-1429-43c8-91dd-574a1a1b5ecd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.272254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21beb74-1429-43c8-91dd-574a1a1b5ecd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.272658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21beb74-1429-43c8-91dd-574a1a1b5ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.301386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e21beb74-1429-43c8-91dd-574a1a1b5ecd None None] MFA Rules not processed for user `9c6e4c7f415643429e54f1d1897b6b47`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:27.302677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e21beb74-1429-43c8-91dd-574a1a1b5ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:27.368512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 994/1963] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:27.372956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21beb74-1429-43c8-91dd-574a1a1b5ecd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:27.434011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21beb74-1429-43c8-91dd-574a1a1b5ecd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:27.438615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4e612-f1fa-4298-87bf-a0678a0f66e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:27.439276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4e612-f1fa-4298-87bf-a0678a0f66e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.439562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4e612-f1fa-4298-87bf-a0678a0f66e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.439864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4e612-f1fa-4298-87bf-a0678a0f66e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.440564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4e612-f1fa-4298-87bf-a0678a0f66e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:27.442546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4e612-f1fa-4298-87bf-a0678a0f66e6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:27.443963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=2,keystone:INSERT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:27.535875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49fadeeabfec4aadaa4acf1a6d02e241'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.536545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49fadeeabfec4aadaa4acf1a6d02e241'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.550196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49fadeeabfec4aadaa4acf1a6d02e241'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.557642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49fadeeabfec4aadaa4acf1a6d02e241'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.589712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 995/1964] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:27 2022] POST /identity/v3/projects => generated 385 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:27.590468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4e612-f1fa-4298-87bf-a0678a0f66e6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:27.592053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c3f4e612-f1fa-4298-87bf-a0678a0f66e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:27.657589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4e612-f1fa-4298-87bf-a0678a0f66e6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:27.662504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:27.664759 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.665060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.665341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] PATH_INFO: `/v3/projects/b4991062b0ee46f3a1be8482be7ef5df` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.667628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4991062b0ee46f3a1be8482be7ef5df)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:27.669844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:27.724612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49fadeeabfec4aadaa4acf1a6d02e241'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.731372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 970/1965] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:27 2022] DELETE /identity/v3/users/49fadeeabfec4aadaa4acf1a6d02e241 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:27.732323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:27.734036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:27.797415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3034b2-e1f0-4ccc-93ce-484181399889 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:27.799738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4991062b0ee46f3a1be8482be7ef5df', 'request_id': 'req-251685e7-4715-48b1-98a7-b6c90ad6bb9b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.805226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4991062b0ee46f3a1be8482be7ef5df', 'request_id': 'req-251685e7-4715-48b1-98a7-b6c90ad6bb9b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.856204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8576a5f-90e7-4758-ac29-4a6c43085f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:27.856946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8576a5f-90e7-4758-ac29-4a6c43085f8e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.857500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8576a5f-90e7-4758-ac29-4a6c43085f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.857782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8576a5f-90e7-4758-ac29-4a6c43085f8e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.858534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8576a5f-90e7-4758-ac29-4a6c43085f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:27.860465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8576a5f-90e7-4758-ac29-4a6c43085f8e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:27.879711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4991062b0ee46f3a1be8482be7ef5df', 'request_id': 'req-251685e7-4715-48b1-98a7-b6c90ad6bb9b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.892048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] The token cache is being invalidate because project b4991062b0ee46f3a1be8482be7ef5df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:27.892359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.893001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-251685e7-4715-48b1-98a7-b6c90ad6bb9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:27.894583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 996/1966] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:27 2022] DELETE /identity/v3/projects/b4991062b0ee46f3a1be8482be7ef5df => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:27.895845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6251011-81df-4dc3-bd70-cd2e5dccccae None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.896134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6251011-81df-4dc3-bd70-cd2e5dccccae None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.896413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6251011-81df-4dc3-bd70-cd2e5dccccae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.913807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e6251011-81df-4dc3-bd70-cd2e5dccccae None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:27.915052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e6251011-81df-4dc3-bd70-cd2e5dccccae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:27.922862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 997/1967] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:27.923712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6251011-81df-4dc3-bd70-cd2e5dccccae None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:27.924976 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e6251011-81df-4dc3-bd70-cd2e5dccccae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:27.936530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6251011-81df-4dc3-bd70-cd2e5dccccae None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:27.941220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:27.942060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:27.942343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:27.942618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] PATH_INFO: `/v3/users/9e5eadb19da7460bb7584d213c5b216b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:27.943845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5eadb19da7460bb7584d213c5b216b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:27.945743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:27.990498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 971/1968] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:27 2022] POST /identity/v3/users => generated 293 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:27.991401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8576a5f-90e7-4758-ac29-4a6c43085f8e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:27.992937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d8576a5f-90e7-4758-ac29-4a6c43085f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:28.048587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8576a5f-90e7-4758-ac29-4a6c43085f8e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:28.052486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbae50-7a6c-41d3-9f01-482adb68aad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:28.053256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbae50-7a6c-41d3-9f01-482adb68aad2 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:28.053543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbae50-7a6c-41d3-9f01-482adb68aad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:28.053841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbae50-7a6c-41d3-9f01-482adb68aad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:28.063659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbae50-7a6c-41d3-9f01-482adb68aad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:28.065082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbae50-7a6c-41d3-9f01-482adb68aad2 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:28.074597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 972/1969] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:28.075376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbae50-7a6c-41d3-9f01-482adb68aad2 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:28.076949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4dcbae50-7a6c-41d3-9f01-482adb68aad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:28.090609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbae50-7a6c-41d3-9f01-482adb68aad2 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:28.095387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc1b9e-317f-4191-a8f1-4c6bf977df0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:28.096050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc1b9e-317f-4191-a8f1-4c6bf977df0e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:28.096333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc1b9e-317f-4191-a8f1-4c6bf977df0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:28.096650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc1b9e-317f-4191-a8f1-4c6bf977df0e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:28.097479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc1b9e-317f-4191-a8f1-4c6bf977df0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:28.099130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc1b9e-317f-4191-a8f1-4c6bf977df0e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:28.110675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 973/1970] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:28.114955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc1b9e-317f-4191-a8f1-4c6bf977df0e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:28.123853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc1b9e-317f-4191-a8f1-4c6bf977df0e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:28.128519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a7521-5669-4a29-a23c-e0c9d5ee8e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:28.129169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a7521-5669-4a29-a23c-e0c9d5ee8e7f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:28.129455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a7521-5669-4a29-a23c-e0c9d5ee8e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:28.129735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a7521-5669-4a29-a23c-e0c9d5ee8e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:28.130486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a7521-5669-4a29-a23c-e0c9d5ee8e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:28.132172 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a7521-5669-4a29-a23c-e0c9d5ee8e7f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:28.143003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 974/1971] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:28.146901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a7521-5669-4a29-a23c-e0c9d5ee8e7f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:28.158168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a7521-5669-4a29-a23c-e0c9d5ee8e7f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:28.163561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dfd1f-8ff4-44ca-8043-89357f455854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:28.164547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dfd1f-8ff4-44ca-8043-89357f455854 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:28.169370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dfd1f-8ff4-44ca-8043-89357f455854 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:28.169370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dfd1f-8ff4-44ca-8043-89357f455854 admin admin] PATH_INFO: `/v3/projects/f3bfadc5233c49278573688ea6f063da/users/6d5e6cd4f3af4255b97dd0dcab625b59/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:28.182969 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dfd1f-8ff4-44ca-8043-89357f455854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3bfadc5233c49278573688ea6f063da, user_id=6d5e6cd4f3af4255b97dd0dcab625b59, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:28.185661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dfd1f-8ff4-44ca-8043-89357f455854 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:28.211161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5eadb19da7460bb7584d213c5b216b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:28.212346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5eadb19da7460bb7584d213c5b216b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:28.220353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5eadb19da7460bb7584d213c5b216b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:28.233378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5eadb19da7460bb7584d213c5b216b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:28.252583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:28.316363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 975/1972] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:28 2022] PUT /identity/v3/projects/f3bfadc5233c49278573688ea6f063da/users/6d5e6cd4f3af4255b97dd0dcab625b59/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:28.327611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d370f67-424c-4917-91a5-a73b84269db5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:28.327901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d370f67-424c-4917-91a5-a73b84269db5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:28.328180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d370f67-424c-4917-91a5-a73b84269db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:28.345043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5eadb19da7460bb7584d213c5b216b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:28.356588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 998/1973] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:27 2022] DELETE /identity/v3/users/9e5eadb19da7460bb7584d213c5b216b => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:28.381352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3d370f67-424c-4917-91a5-a73b84269db5 None None] MFA Rules not processed for user `6d5e6cd4f3af4255b97dd0dcab625b59`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:28.382707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3d370f67-424c-4917-91a5-a73b84269db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:28.390434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:28.455287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 976/1974] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:28.459834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:28.461341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:28.529643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592dda7c-a1c6-4598-8b9d-2281b1c8f837 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:28.536228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4e507-70dd-4b72-9672-2ba9d3f6312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:28.537412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e507-70dd-4b72-9672-2ba9d3f6312a admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:28.537702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e507-70dd-4b72-9672-2ba9d3f6312a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:28.538017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4e507-70dd-4b72-9672-2ba9d3f6312a admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:28.538902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4e507-70dd-4b72-9672-2ba9d3f6312a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:28.541493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4e507-70dd-4b72-9672-2ba9d3f6312a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:28.662790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 999/1975] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:28 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:28.667987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d370f67-424c-4917-91a5-a73b84269db5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:28.695543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d370f67-424c-4917-91a5-a73b84269db5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:28.761531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfaf8d-1fcd-4aae-bb71-8c959d40ac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:28.762590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfaf8d-1fcd-4aae-bb71-8c959d40ac8e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:28.762945 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfaf8d-1fcd-4aae-bb71-8c959d40ac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:28.763731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfaf8d-1fcd-4aae-bb71-8c959d40ac8e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:28.764480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfaf8d-1fcd-4aae-bb71-8c959d40ac8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:28.766445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfaf8d-1fcd-4aae-bb71-8c959d40ac8e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:28.804954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4e507-70dd-4b72-9672-2ba9d3f6312a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:28.808911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9aa4e507-70dd-4b72-9672-2ba9d3f6312a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:28.829515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4e507-70dd-4b72-9672-2ba9d3f6312a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:28.836796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:28.839633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:28.842396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:28.843152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] PATH_INFO: `/v3/projects/25f510accfb348c09e5310ad8077905a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:28.845363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f510accfb348c09e5310ad8077905a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:28.847954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:28.929609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 977/1976] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:28 2022] POST /identity/v3/users => generated 299 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:28.933594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfaf8d-1fcd-4aae-bb71-8c959d40ac8e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:28.981680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfaf8d-1fcd-4aae-bb71-8c959d40ac8e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:28.988110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f510accfb348c09e5310ad8077905a', 'request_id': 'req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:28.994385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f510accfb348c09e5310ad8077905a', 'request_id': 'req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.043440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a88a2-81c0-41ef-b901-9b85b5638131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:29.044132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a88a2-81c0-41ef-b901-9b85b5638131 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:29.044417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a88a2-81c0-41ef-b901-9b85b5638131 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:29.044712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a88a2-81c0-41ef-b901-9b85b5638131 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:29.045500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a88a2-81c0-41ef-b901-9b85b5638131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:29.047178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a88a2-81c0-41ef-b901-9b85b5638131 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:29.059870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f510accfb348c09e5310ad8077905a', 'request_id': 'req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.060981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 978/1977] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:29.065066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a88a2-81c0-41ef-b901-9b85b5638131 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:29.076203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] The token cache is being invalidate because project 25f510accfb348c09e5310ad8077905a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:29.076830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.077752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a88a2-81c0-41ef-b901-9b85b5638131 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:29.078046 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.084647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1000/1978] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:28 2022] DELETE /identity/v3/projects/25f510accfb348c09e5310ad8077905a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:29.085687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:29.087325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:29.091581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff541-d9fb-426a-839e-b64345f952a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:29.097316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff541-d9fb-426a-839e-b64345f952a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:29.097316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff541-d9fb-426a-839e-b64345f952a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:29.097316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff541-d9fb-426a-839e-b64345f952a6 admin admin] PATH_INFO: `/v3/projects/2840df1db5764778ae9a22ba3dea99b3/users/c8432f7513774619b4cb3d4742cdff0a/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:29.110634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acddc6-e0e7-47bd-9541-9bbcc95cf6d5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:29.112690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff541-d9fb-426a-839e-b64345f952a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2840df1db5764778ae9a22ba3dea99b3, user_id=c8432f7513774619b4cb3d4742cdff0a, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:29.115657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff541-d9fb-426a-839e-b64345f952a6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:29.116650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:29.117612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:29.118073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:29.126319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] PATH_INFO: `/v3/users/bfe0f3bf5eaa4d5d959efcb2d967ccf7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:29.130676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfe0f3bf5eaa4d5d959efcb2d967ccf7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:29.132811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:29.185633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 979/1979] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:29 2022] PUT /identity/v3/projects/2840df1db5764778ae9a22ba3dea99b3/users/c8432f7513774619b4cb3d4742cdff0a/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:29.188103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff541-d9fb-426a-839e-b64345f952a6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:29.188199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1d2ff541-d9fb-426a-839e-b64345f952a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:29.257139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff541-d9fb-426a-839e-b64345f952a6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:29.263619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:29.265092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:29.265388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:29.265663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] PATH_INFO: `/v3/users/d2ed7dd8d03c4c42925e3efca1915a46` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:29.267059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ed7dd8d03c4c42925e3efca1915a46)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:29.269068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:29.527357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfe0f3bf5eaa4d5d959efcb2d967ccf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.527972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfe0f3bf5eaa4d5d959efcb2d967ccf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.534722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfe0f3bf5eaa4d5d959efcb2d967ccf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.546985 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfe0f3bf5eaa4d5d959efcb2d967ccf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.556069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:29.680351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfe0f3bf5eaa4d5d959efcb2d967ccf7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.688267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1001/1980] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:29 2022] DELETE /identity/v3/users/bfe0f3bf5eaa4d5d959efcb2d967ccf7 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:29.692235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:29.693515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:29.695309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ed7dd8d03c4c42925e3efca1915a46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.695921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ed7dd8d03c4c42925e3efca1915a46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.709799 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ed7dd8d03c4c42925e3efca1915a46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.717734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ed7dd8d03c4c42925e3efca1915a46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.769060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae1b18-be20-4cb2-b7ca-7f1c0215aaf3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:29.775337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced061-ea44-4278-99c9-36e369478248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:29.776089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced061-ea44-4278-99c9-36e369478248 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:29.776308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced061-ea44-4278-99c9-36e369478248 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:29.776591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced061-ea44-4278-99c9-36e369478248 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:29.777591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccced061-ea44-4278-99c9-36e369478248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:29.779001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccced061-ea44-4278-99c9-36e369478248 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:29.788271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ed7dd8d03c4c42925e3efca1915a46'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:29.793773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1002/1981] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:29.794664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced061-ea44-4278-99c9-36e369478248 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:29.797391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ccced061-ea44-4278-99c9-36e369478248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:29.797936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 980/1982] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:29 2022] DELETE /identity/v3/users/d2ed7dd8d03c4c42925e3efca1915a46 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:29.799089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:29.800966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:29.818542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced061-ea44-4278-99c9-36e369478248 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:29.829671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:29.882573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:29.883449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:29.883870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:29.884156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] PATH_INFO: `/v3/projects/6c11f9aab8664daa8ea171cf44ccd571` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:29.885452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c11f9aab8664daa8ea171cf44ccd571)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:29.887822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:29.911651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f185093-1447-4839-a6a0-bdf5fdaef0a8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:29.918889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4f5f9-4d40-463b-9495-b23caf59ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:29.920097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4f5f9-4d40-463b-9495-b23caf59ce59 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:29.920808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4f5f9-4d40-463b-9495-b23caf59ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:29.921091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4f5f9-4d40-463b-9495-b23caf59ce59 admin admin] PATH_INFO: `/v3/projects/2840df1db5764778ae9a22ba3dea99b3/users/c8432f7513774619b4cb3d4742cdff0a/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:29.925137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4f5f9-4d40-463b-9495-b23caf59ce59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2840df1db5764778ae9a22ba3dea99b3, user_id=c8432f7513774619b4cb3d4742cdff0a, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:29.927950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4f5f9-4d40-463b-9495-b23caf59ce59 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:29.940201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 981/1983] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:29 2022] PUT /identity/v3/projects/2840df1db5764778ae9a22ba3dea99b3/users/c8432f7513774619b4cb3d4742cdff0a/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:29.952351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77270fb3-1b7f-4f35-97c5-b7d72d80bba9 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:29.952857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77270fb3-1b7f-4f35-97c5-b7d72d80bba9 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:29.953143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77270fb3-1b7f-4f35-97c5-b7d72d80bba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:29.982146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-77270fb3-1b7f-4f35-97c5-b7d72d80bba9 None None] MFA Rules not processed for user `c8432f7513774619b4cb3d4742cdff0a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:29.983367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-77270fb3-1b7f-4f35-97c5-b7d72d80bba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:30.055672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c11f9aab8664daa8ea171cf44ccd571', 'request_id': 'req-a82e30c3-a6bf-4343-ba19-61541e1bd170'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.062928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c11f9aab8664daa8ea171cf44ccd571', 'request_id': 'req-a82e30c3-a6bf-4343-ba19-61541e1bd170'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.071097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 982/1984] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:30.085113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77270fb3-1b7f-4f35-97c5-b7d72d80bba9 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:30.086682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-77270fb3-1b7f-4f35-97c5-b7d72d80bba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:30.147049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77270fb3-1b7f-4f35-97c5-b7d72d80bba9 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:30.161792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c11f9aab8664daa8ea171cf44ccd571', 'request_id': 'req-a82e30c3-a6bf-4343-ba19-61541e1bd170'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.180615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] The token cache is being invalidate because project 6c11f9aab8664daa8ea171cf44ccd571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:30.180991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.182888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a82e30c3-a6bf-4343-ba19-61541e1bd170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.188521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1003/1985] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:29 2022] DELETE /identity/v3/projects/6c11f9aab8664daa8ea171cf44ccd571 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:30.191658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6819878-0bfa-4a41-aa57-4958be222292 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:30.192175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6819878-0bfa-4a41-aa57-4958be222292 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:30.192653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6819878-0bfa-4a41-aa57-4958be222292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:30.205189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:30.205798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:30.206199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:30.206483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:30.208361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:30.227897 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e6819878-0bfa-4a41-aa57-4958be222292 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:30.229125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e6819878-0bfa-4a41-aa57-4958be222292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:30.275225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:30.276772 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:30.287087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 983/1986] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:30.288131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:30.289620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:30.312803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1004/1987] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:30.321012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc793a4-9622-41fa-875f-f55fa2cce94e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:30.321526 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc793a4-9622-41fa-875f-f55fa2cce94e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:30.322023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc793a4-9622-41fa-875f-f55fa2cce94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:30.341683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0fc793a4-9622-41fa-875f-f55fa2cce94e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:30.343491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0fc793a4-9622-41fa-875f-f55fa2cce94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:30.353207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1005/1988] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:30.357652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc793a4-9622-41fa-875f-f55fa2cce94e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:30.368363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc793a4-9622-41fa-875f-f55fa2cce94e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:30.374014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb7fe1-936c-4e24-824e-497c382c9f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:30.374974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb7fe1-936c-4e24-824e-497c382c9f25 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:30.375408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb7fe1-936c-4e24-824e-497c382c9f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:30.375833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb7fe1-936c-4e24-824e-497c382c9f25 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:30.376728 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb7fe1-936c-4e24-824e-497c382c9f25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:30.377182 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb7fe1-936c-4e24-824e-497c382c9f25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:30.379027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb7fe1-936c-4e24-824e-497c382c9f25 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:30.379298 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d704603-9e3b-4f02-aa9a-0650ad38c442 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:30.385561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:30.386841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:30.387812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:30.387812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] PATH_INFO: `/v3/projects/69776fa5af0c4573b668bbf8dc200931` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:30.389684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=69776fa5af0c4573b668bbf8dc200931)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:30.392005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:30.395613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1006/1989] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:59:30.399395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb7fe1-936c-4e24-824e-497c382c9f25 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:30.409940 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb7fe1-936c-4e24-824e-497c382c9f25 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:30.424807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d37a7a-7f37-4e87-8ad3-f58660274fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:30.425711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d37a7a-7f37-4e87-8ad3-f58660274fde admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:30.428070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d37a7a-7f37-4e87-8ad3-f58660274fde admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:30.428531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d37a7a-7f37-4e87-8ad3-f58660274fde admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:30.429256 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d37a7a-7f37-4e87-8ad3-f58660274fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:30.431395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d37a7a-7f37-4e87-8ad3-f58660274fde admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:30.551510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69776fa5af0c4573b668bbf8dc200931', 'request_id': 'req-10833b05-8155-4be0-8bc5-a770a74631ed'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.556932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69776fa5af0c4573b668bbf8dc200931', 'request_id': 'req-10833b05-8155-4be0-8bc5-a770a74631ed'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.585775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1007/1990] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:30 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:30.586807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d37a7a-7f37-4e87-8ad3-f58660274fde admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:30.588861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-04d37a7a-7f37-4e87-8ad3-f58660274fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:30.658452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d37a7a-7f37-4e87-8ad3-f58660274fde admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:30.663302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:30.664615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69776fa5af0c4573b668bbf8dc200931', 'request_id': 'req-10833b05-8155-4be0-8bc5-a770a74631ed'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.669953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:30.670315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:30.670613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] PATH_INFO: `/v3/projects/7a37c4c51d3e4db288867fd374d95d5b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:30.672173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a37c4c51d3e4db288867fd374d95d5b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:30.674455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:30.677117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] The token cache is being invalidate because project 69776fa5af0c4573b668bbf8dc200931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:30.677540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.678116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.687520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 984/1991] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:30 2022] DELETE /identity/v3/projects/69776fa5af0c4573b668bbf8dc200931 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:30.692290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:30.693870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:30.775109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10833b05-8155-4be0-8bc5-a770a74631ed admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:30.779951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2eb189-9dd9-43ce-96ce-16c64a0e781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:30.780679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eb189-9dd9-43ce-96ce-16c64a0e781c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:30.780962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eb189-9dd9-43ce-96ce-16c64a0e781c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:30.781241 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eb189-9dd9-43ce-96ce-16c64a0e781c admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:30.781966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2eb189-9dd9-43ce-96ce-16c64a0e781c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:30.783896 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2eb189-9dd9-43ce-96ce-16c64a0e781c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:30.824900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a37c4c51d3e4db288867fd374d95d5b', 'request_id': 'req-9cfce80c-8122-42a4-8081-0a7202e480cf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.830220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a37c4c51d3e4db288867fd374d95d5b', 'request_id': 'req-9cfce80c-8122-42a4-8081-0a7202e480cf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.923318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a37c4c51d3e4db288867fd374d95d5b', 'request_id': 'req-9cfce80c-8122-42a4-8081-0a7202e480cf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.923495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:30.932910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 985/1992] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:30 2022] POST /identity/v3/users => generated 291 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:30.935639 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c5026-5048-4379-a81d-7ac46cdd6adb None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:30.936130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c5026-5048-4379-a81d-7ac46cdd6adb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:30.936437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237c5026-5048-4379-a81d-7ac46cdd6adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:30.950177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] The token cache is being invalidate because project 7a37c4c51d3e4db288867fd374d95d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:30.950473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.951217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:30.952861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1008/1993] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:30 2022] DELETE /identity/v3/projects/7a37c4c51d3e4db288867fd374d95d5b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:30.953624 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:30.954935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:30.972057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-237c5026-5048-4379-a81d-7ac46cdd6adb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:30.973280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-237c5026-5048-4379-a81d-7ac46cdd6adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:31.032077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfce80c-8122-42a4-8081-0a7202e480cf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.039239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 986/1994] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:31.041070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5497b-1166-4c35-9902-59ff7c3d3e86 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.041449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5497b-1166-4c35-9902-59ff7c3d3e86 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.041632 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5497b-1166-4c35-9902-59ff7c3d3e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.048392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328dc83-c3ac-4dfb-949c-0c8a065a7f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.049152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328dc83-c3ac-4dfb-949c-0c8a065a7f4b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.049443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328dc83-c3ac-4dfb-949c-0c8a065a7f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.049720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328dc83-c3ac-4dfb-949c-0c8a065a7f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.050491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328dc83-c3ac-4dfb-949c-0c8a065a7f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.053224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328dc83-c3ac-4dfb-949c-0c8a065a7f4b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.069547 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1009/1995] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:31.075513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f2009-e30d-473d-ac14-9928c530b90e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.075513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f2009-e30d-473d-ac14-9928c530b90e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.075513 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f2009-e30d-473d-ac14-9928c530b90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.075781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7df5497b-1166-4c35-9902-59ff7c3d3e86 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:31.075781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7df5497b-1166-4c35-9902-59ff7c3d3e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:31.075941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:31.085768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 987/1996] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:31.088170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5497b-1166-4c35-9902-59ff7c3d3e86 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.110442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e33f2009-e30d-473d-ac14-9928c530b90e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:31.111866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e33f2009-e30d-473d-ac14-9928c530b90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:31.126059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5497b-1166-4c35-9902-59ff7c3d3e86 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.126157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1010/1997] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:31.126648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021e2e6-a5d2-4249-88ab-0030d3dc75e0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.126984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021e2e6-a5d2-4249-88ab-0030d3dc75e0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.127312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021e2e6-a5d2-4249-88ab-0030d3dc75e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.131049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaf641-2d46-41a5-a4bd-61f3c31f6c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.131780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaf641-2d46-41a5-a4bd-61f3c31f6c13 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.132069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaf641-2d46-41a5-a4bd-61f3c31f6c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.132502 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaf641-2d46-41a5-a4bd-61f3c31f6c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.133230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaf641-2d46-41a5-a4bd-61f3c31f6c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.135488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaf641-2d46-41a5-a4bd-61f3c31f6c13 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.151332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f021e2e6-a5d2-4249-88ab-0030d3dc75e0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:31.152757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f021e2e6-a5d2-4249-88ab-0030d3dc75e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:31.169908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 988/1998] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:31.169908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaf641-2d46-41a5-a4bd-61f3c31f6c13 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.178304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1011/1999] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:31.179477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021e2e6-a5d2-4249-88ab-0030d3dc75e0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.191524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaf641-2d46-41a5-a4bd-61f3c31f6c13 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.192220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021e2e6-a5d2-4249-88ab-0030d3dc75e0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.199667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250eeb17-52ec-4405-a589-235f74457628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.200418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d82aad-eb44-4c30-bbb6-9b286a8e46f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.200868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250eeb17-52ec-4405-a589-235f74457628 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.201612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250eeb17-52ec-4405-a589-235f74457628 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.201779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d82aad-eb44-4c30-bbb6-9b286a8e46f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.202111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d82aad-eb44-4c30-bbb6-9b286a8e46f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.202391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d82aad-eb44-4c30-bbb6-9b286a8e46f1 admin admin] PATH_INFO: `/v3/projects/b291c2cbdcf8476a82a651414daa676f/users/181022a415f7483d8ad1099231d4c5b1/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.202640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250eeb17-52ec-4405-a589-235f74457628 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.203734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250eeb17-52ec-4405-a589-235f74457628 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:31.204193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250eeb17-52ec-4405-a589-235f74457628 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.206089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250eeb17-52ec-4405-a589-235f74457628 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.221364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d82aad-eb44-4c30-bbb6-9b286a8e46f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b291c2cbdcf8476a82a651414daa676f, user_id=181022a415f7483d8ad1099231d4c5b1, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.221583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 989/2000] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:31.225336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d82aad-eb44-4c30-bbb6-9b286a8e46f1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.226558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250eeb17-52ec-4405-a589-235f74457628 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.239268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250eeb17-52ec-4405-a589-235f74457628 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.245416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25fa81-fafd-48ca-a5e8-756ff89a7ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.246423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25fa81-fafd-48ca-a5e8-756ff89a7ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.246853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25fa81-fafd-48ca-a5e8-756ff89a7ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.247143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25fa81-fafd-48ca-a5e8-756ff89a7ee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.247881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25fa81-fafd-48ca-a5e8-756ff89a7ee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:31.248196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25fa81-fafd-48ca-a5e8-756ff89a7ee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.250234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25fa81-fafd-48ca-a5e8-756ff89a7ee1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.262973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 990/2001] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:31.263879 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25fa81-fafd-48ca-a5e8-756ff89a7ee1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.273931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25fa81-fafd-48ca-a5e8-756ff89a7ee1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.279132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a94d9-7bee-4fa6-b804-c6af70f56eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.279769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a94d9-7bee-4fa6-b804-c6af70f56eca admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.280220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a94d9-7bee-4fa6-b804-c6af70f56eca admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.280545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a94d9-7bee-4fa6-b804-c6af70f56eca admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.281246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a94d9-7bee-4fa6-b804-c6af70f56eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.283225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a94d9-7bee-4fa6-b804-c6af70f56eca admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.319827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1012/2002] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:31 2022] PUT /identity/v3/projects/b291c2cbdcf8476a82a651414daa676f/users/181022a415f7483d8ad1099231d4c5b1/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:31.320827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d82aad-eb44-4c30-bbb6-9b286a8e46f1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.393912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d82aad-eb44-4c30-bbb6-9b286a8e46f1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.398949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5d55a-38b4-4450-a7b8-b01c67b78b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.399664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5d55a-38b4-4450-a7b8-b01c67b78b0a admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.399952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5d55a-38b4-4450-a7b8-b01c67b78b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.400792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5d55a-38b4-4450-a7b8-b01c67b78b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.401489 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5d55a-38b4-4450-a7b8-b01c67b78b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.403454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5d55a-38b4-4450-a7b8-b01c67b78b0a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.434036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 991/2003] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:31.435665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1187566-a8e8-487a-89ca-4d82128d7100 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.436832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1187566-a8e8-487a-89ca-4d82128d7100 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.437478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1187566-a8e8-487a-89ca-4d82128d7100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.470775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d1187566-a8e8-487a-89ca-4d82128d7100 None None] MFA Rules not processed for user `181022a415f7483d8ad1099231d4c5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:31.472144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d1187566-a8e8-487a-89ca-4d82128d7100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:31.538217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 992/2004] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:31.539396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1187566-a8e8-487a-89ca-4d82128d7100 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.545222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1013/2005] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:31.548823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5d55a-38b4-4450-a7b8-b01c67b78b0a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.612521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1187566-a8e8-487a-89ca-4d82128d7100 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.617679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28947f-1166-4721-9ac0-40850361a5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.618447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28947f-1166-4721-9ac0-40850361a5ef admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.618556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5d55a-38b4-4450-a7b8-b01c67b78b0a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.619530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28947f-1166-4721-9ac0-40850361a5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.619607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28947f-1166-4721-9ac0-40850361a5ef admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.620415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28947f-1166-4721-9ac0-40850361a5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.622443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28947f-1166-4721-9ac0-40850361a5ef admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.623945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79296f-b3bd-4af9-82c8-bed28bb33579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.624608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79296f-b3bd-4af9-82c8-bed28bb33579 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.624892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79296f-b3bd-4af9-82c8-bed28bb33579 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.625221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79296f-b3bd-4af9-82c8-bed28bb33579 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.625935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79296f-b3bd-4af9-82c8-bed28bb33579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.628330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79296f-b3bd-4af9-82c8-bed28bb33579 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.777384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 993/2006] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/users => generated 294 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:31.778490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28947f-1166-4721-9ac0-40850361a5ef admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.788086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28947f-1166-4721-9ac0-40850361a5ef admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.792844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a3ba2-9aef-4bcc-a1f6-0e450dfb0a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.793489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a3ba2-9aef-4bcc-a1f6-0e450dfb0a46 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.793772 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a3ba2-9aef-4bcc-a1f6-0e450dfb0a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.794087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a3ba2-9aef-4bcc-a1f6-0e450dfb0a46 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.794757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a3ba2-9aef-4bcc-a1f6-0e450dfb0a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.796676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a3ba2-9aef-4bcc-a1f6-0e450dfb0a46 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.816570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1014/2007] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/projects => generated 381 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:31.816570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79296f-b3bd-4af9-82c8-bed28bb33579 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.878314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79296f-b3bd-4af9-82c8-bed28bb33579 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.883025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c88c4-ab31-4b29-a603-48453782efe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.883694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c88c4-ab31-4b29-a603-48453782efe7 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.883979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c88c4-ab31-4b29-a603-48453782efe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.884257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c88c4-ab31-4b29-a603-48453782efe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.885637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c88c4-ab31-4b29-a603-48453782efe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.887528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c88c4-ab31-4b29-a603-48453782efe7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.897033 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1015/2008] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:31.898001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c88c4-ab31-4b29-a603-48453782efe7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.907215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c88c4-ab31-4b29-a603-48453782efe7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.911736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615984d2-83e1-4477-ab19-298f6de16e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.912342 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615984d2-83e1-4477-ab19-298f6de16e74 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.912628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615984d2-83e1-4477-ab19-298f6de16e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.912905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615984d2-83e1-4477-ab19-298f6de16e74 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.913597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615984d2-83e1-4477-ab19-298f6de16e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.915457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615984d2-83e1-4477-ab19-298f6de16e74 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.916389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:31.956665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 994/2009] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:31.957627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a3ba2-9aef-4bcc-a1f6-0e450dfb0a46 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.967196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a3ba2-9aef-4bcc-a1f6-0e450dfb0a46 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:31.972182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3889b-0226-4084-abe3-37cb9a3f3603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:31.972836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3889b-0226-4084-abe3-37cb9a3f3603 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:31.973117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3889b-0226-4084-abe3-37cb9a3f3603 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:31.973392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3889b-0226-4084-abe3-37cb9a3f3603 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:31.974139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3889b-0226-4084-abe3-37cb9a3f3603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:31.975750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3889b-0226-4084-abe3-37cb9a3f3603 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:31.987303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 995/2010] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:31.988069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3889b-0226-4084-abe3-37cb9a3f3603 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:31.997924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3889b-0226-4084-abe3-37cb9a3f3603 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.002380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91afe8f-d4b8-4d81-b1f7-4a1b5cd9b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.003039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91afe8f-d4b8-4d81-b1f7-4a1b5cd9b903 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.003319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91afe8f-d4b8-4d81-b1f7-4a1b5cd9b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.003593 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91afe8f-d4b8-4d81-b1f7-4a1b5cd9b903 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.005902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91afe8f-d4b8-4d81-b1f7-4a1b5cd9b903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.005902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91afe8f-d4b8-4d81-b1f7-4a1b5cd9b903 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.016075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 996/2011] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:32.016800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91afe8f-d4b8-4d81-b1f7-4a1b5cd9b903 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.026199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91afe8f-d4b8-4d81-b1f7-4a1b5cd9b903 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.030949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fa7e0-1438-4acb-81ea-84aafbd8845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.031783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fa7e0-1438-4acb-81ea-84aafbd8845e admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.032099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fa7e0-1438-4acb-81ea-84aafbd8845e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.032377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fa7e0-1438-4acb-81ea-84aafbd8845e admin admin] PATH_INFO: `/v3/projects/91de1e358d6b4adbafa953bbbd3d5e52/users/9b551f2e38d84db6985ea75d32d7acf2/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.046497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1fa7e0-1438-4acb-81ea-84aafbd8845e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91de1e358d6b4adbafa953bbbd3d5e52, user_id=9b551f2e38d84db6985ea75d32d7acf2, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.049116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1fa7e0-1438-4acb-81ea-84aafbd8845e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.085118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1016/2012] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:31 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:32.085971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615984d2-83e1-4477-ab19-298f6de16e74 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.095511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615984d2-83e1-4477-ab19-298f6de16e74 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.100163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c19f3-cc0b-493c-927d-0492956f38c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.100847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c19f3-cc0b-493c-927d-0492956f38c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.101131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c19f3-cc0b-493c-927d-0492956f38c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.101413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c19f3-cc0b-493c-927d-0492956f38c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.102187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c19f3-cc0b-493c-927d-0492956f38c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.103808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c19f3-cc0b-493c-927d-0492956f38c3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.113596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1017/2013] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:32.114519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c19f3-cc0b-493c-927d-0492956f38c3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.124585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c19f3-cc0b-493c-927d-0492956f38c3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.129289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a952c76-f2af-4805-b0e5-a61a8540d531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.129899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a952c76-f2af-4805-b0e5-a61a8540d531 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.130471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a952c76-f2af-4805-b0e5-a61a8540d531 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.130795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a952c76-f2af-4805-b0e5-a61a8540d531 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.131697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a952c76-f2af-4805-b0e5-a61a8540d531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.133799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a952c76-f2af-4805-b0e5-a61a8540d531 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.145216 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1018/2014] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:32.146697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a952c76-f2af-4805-b0e5-a61a8540d531 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.155625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a952c76-f2af-4805-b0e5-a61a8540d531 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.160507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266bdab-76b4-4170-ad71-a2a04e10efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.161333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266bdab-76b4-4170-ad71-a2a04e10efe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.161614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266bdab-76b4-4170-ad71-a2a04e10efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.161908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266bdab-76b4-4170-ad71-a2a04e10efe5 admin admin] PATH_INFO: `/v3/projects/dc646fd36ff140bc9e3cae95da69b1c0/users/99d62013c90b4e29ab3e64ae7c85e6c0/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.176225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266bdab-76b4-4170-ad71-a2a04e10efe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc646fd36ff140bc9e3cae95da69b1c0, user_id=99d62013c90b4e29ab3e64ae7c85e6c0, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.186928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 997/2015] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:32 2022] PUT /identity/v3/projects/91de1e358d6b4adbafa953bbbd3d5e52/users/9b551f2e38d84db6985ea75d32d7acf2/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:32.186928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fa7e0-1438-4acb-81ea-84aafbd8845e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.187480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266bdab-76b4-4170-ad71-a2a04e10efe5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.248834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fa7e0-1438-4acb-81ea-84aafbd8845e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.254011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd85f3e-cdc8-4aab-a188-2aa1ac1b8073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.255063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd85f3e-cdc8-4aab-a188-2aa1ac1b8073 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.255272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd85f3e-cdc8-4aab-a188-2aa1ac1b8073 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.255556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd85f3e-cdc8-4aab-a188-2aa1ac1b8073 admin admin] PATH_INFO: `/v3/projects/fa08ecb75f6b48768741d9d27a0263fb/users/bca5fde7191c4f76b8489caf62483d3d/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.270178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd85f3e-cdc8-4aab-a188-2aa1ac1b8073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa08ecb75f6b48768741d9d27a0263fb, user_id=bca5fde7191c4f76b8489caf62483d3d, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.272849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd85f3e-cdc8-4aab-a188-2aa1ac1b8073 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.354798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1019/2016] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:32 2022] PUT /identity/v3/projects/dc646fd36ff140bc9e3cae95da69b1c0/users/99d62013c90b4e29ab3e64ae7c85e6c0/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:32.355884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15c62d-dc88-4924-88b6-bbb61ed627a3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.356252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15c62d-dc88-4924-88b6-bbb61ed627a3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.356684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15c62d-dc88-4924-88b6-bbb61ed627a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.390352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 998/2017] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:32 2022] PUT /identity/v3/projects/fa08ecb75f6b48768741d9d27a0263fb/users/bca5fde7191c4f76b8489caf62483d3d/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:32.393541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ef15c62d-dc88-4924-88b6-bbb61ed627a3 None None] MFA Rules not processed for user `9b551f2e38d84db6985ea75d32d7acf2`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:32.393655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded78d4-7b02-49cc-818a-6780548cb816 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.393655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded78d4-7b02-49cc-818a-6780548cb816 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.393655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded78d4-7b02-49cc-818a-6780548cb816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.393906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ef15c62d-dc88-4924-88b6-bbb61ed627a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:32.438729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-cded78d4-7b02-49cc-818a-6780548cb816 None None] MFA Rules not processed for user `99d62013c90b4e29ab3e64ae7c85e6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:32.438729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cded78d4-7b02-49cc-818a-6780548cb816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:32.494025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1020/2018] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:32.494025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15c62d-dc88-4924-88b6-bbb61ed627a3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.527835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 999/2019] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:32.528520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded78d4-7b02-49cc-818a-6780548cb816 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.541204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:32.571012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15c62d-dc88-4924-88b6-bbb61ed627a3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.576516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd6bc9-af69-481b-a5a5-907526fea66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.577435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6bc9-af69-481b-a5a5-907526fea66a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.577704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6bc9-af69-481b-a5a5-907526fea66a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.578114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6bc9-af69-481b-a5a5-907526fea66a admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.578969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd6bc9-af69-481b-a5a5-907526fea66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.580719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd6bc9-af69-481b-a5a5-907526fea66a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.592515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1021/2020] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:32.593569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd6bc9-af69-481b-a5a5-907526fea66a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.596840 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0bdd6bc9-af69-481b-a5a5-907526fea66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:32.598993 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded78d4-7b02-49cc-818a-6780548cb816 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.604668 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4876-628e-4bf3-a3dd-d14ce75d772b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.605311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4876-628e-4bf3-a3dd-d14ce75d772b admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.605546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4876-628e-4bf3-a3dd-d14ce75d772b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.605924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc4876-628e-4bf3-a3dd-d14ce75d772b admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.606643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc4876-628e-4bf3-a3dd-d14ce75d772b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.608625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc4876-628e-4bf3-a3dd-d14ce75d772b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.657591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd6bc9-af69-481b-a5a5-907526fea66a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.661511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef4ae8-3d27-40fb-9de2-40079d0ebc26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.662207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef4ae8-3d27-40fb-9de2-40079d0ebc26 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.662482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef4ae8-3d27-40fb-9de2-40079d0ebc26 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.662767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef4ae8-3d27-40fb-9de2-40079d0ebc26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.664656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-51ef4ae8-3d27-40fb-9de2-40079d0ebc26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:32.738172 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1000/2021] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:32 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:32.739187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4876-628e-4bf3-a3dd-d14ce75d772b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.742188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef4ae8-3d27-40fb-9de2-40079d0ebc26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.743599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef4ae8-3d27-40fb-9de2-40079d0ebc26 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.760721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1022/2022] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:32.761745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef4ae8-3d27-40fb-9de2-40079d0ebc26 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.778189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef4ae8-3d27-40fb-9de2-40079d0ebc26 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.809168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc4876-628e-4bf3-a3dd-d14ce75d772b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.814072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fa0fa-3e22-4678-b273-609df82d3ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.814747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fa0fa-3e22-4678-b273-609df82d3ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.815063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fa0fa-3e22-4678-b273-609df82d3ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.815344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fa0fa-3e22-4678-b273-609df82d3ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.816189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fa0fa-3e22-4678-b273-609df82d3ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.818867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fa0fa-3e22-4678-b273-609df82d3ee3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.846486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b71b9-d984-41cf-9346-869beacb5bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.848254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b71b9-d984-41cf-9346-869beacb5bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.848698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b71b9-d984-41cf-9346-869beacb5bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.849226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b71b9-d984-41cf-9346-869beacb5bd3 admin admin] PATH_INFO: `/v3/projects/fa08ecb75f6b48768741d9d27a0263fb/users/bca5fde7191c4f76b8489caf62483d3d/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.851332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b71b9-d984-41cf-9346-869beacb5bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa08ecb75f6b48768741d9d27a0263fb, user_id=bca5fde7191c4f76b8489caf62483d3d, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.854128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b71b9-d984-41cf-9346-869beacb5bd3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:32.865304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1023/2023] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:32 2022] PUT /identity/v3/projects/fa08ecb75f6b48768741d9d27a0263fb/users/bca5fde7191c4f76b8489caf62483d3d/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:32.866244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b71b9-d984-41cf-9346-869beacb5bd3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:32.911049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1001/2024] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:32 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:32.913031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce87e0-2b82-46a4-a762-a549a27be367 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.913325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce87e0-2b82-46a4-a762-a549a27be367 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.913652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce87e0-2b82-46a4-a762-a549a27be367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.932271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:32.937512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b71b9-d984-41cf-9346-869beacb5bd3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:32.942950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fcce87e0-2b82-46a4-a762-a549a27be367 None None] MFA Rules not processed for user `bca5fde7191c4f76b8489caf62483d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:32.944240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fcce87e0-2b82-46a4-a762-a549a27be367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:32.995903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d84b2-3759-4a75-a53f-f0d9205afdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:32.996585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d84b2-3759-4a75-a53f-f0d9205afdd5 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:32.996872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d84b2-3759-4a75-a53f-f0d9205afdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:32.997154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d84b2-3759-4a75-a53f-f0d9205afdd5 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:32.997845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d84b2-3759-4a75-a53f-f0d9205afdd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:32.999800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d84b2-3759-4a75-a53f-f0d9205afdd5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.014274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1002/2025] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:33.015091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce87e0-2b82-46a4-a762-a549a27be367 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.024804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce87e0-2b82-46a4-a762-a549a27be367 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.029520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5dc8e-8a77-410b-bcf0-06690fb21728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.030210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5dc8e-8a77-410b-bcf0-06690fb21728 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.030498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5dc8e-8a77-410b-bcf0-06690fb21728 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.032303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5dc8e-8a77-410b-bcf0-06690fb21728 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.032303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5dc8e-8a77-410b-bcf0-06690fb21728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.033481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5dc8e-8a77-410b-bcf0-06690fb21728 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.145113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1024/2026] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:32 2022] POST /identity/v3/users => generated 300 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:33.145946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d84b2-3759-4a75-a53f-f0d9205afdd5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.147578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-791d84b2-3759-4a75-a53f-f0d9205afdd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:33.148424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1003/2027] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:33 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:33.149318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5dc8e-8a77-410b-bcf0-06690fb21728 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.158967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5dc8e-8a77-410b-bcf0-06690fb21728 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.163873 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d829a-72ab-4959-88aa-96643daf129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.164559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d829a-72ab-4959-88aa-96643daf129d admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.164872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d829a-72ab-4959-88aa-96643daf129d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.165151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d829a-72ab-4959-88aa-96643daf129d admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.166451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d829a-72ab-4959-88aa-96643daf129d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.168063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d829a-72ab-4959-88aa-96643daf129d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.178072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1004/2028] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:33.178866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d829a-72ab-4959-88aa-96643daf129d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.188756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d829a-72ab-4959-88aa-96643daf129d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.193518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905e0ee-8d3d-4c56-a92d-296dd76101d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.194198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905e0ee-8d3d-4c56-a92d-296dd76101d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.194484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905e0ee-8d3d-4c56-a92d-296dd76101d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.194762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905e0ee-8d3d-4c56-a92d-296dd76101d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.195465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905e0ee-8d3d-4c56-a92d-296dd76101d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.197057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905e0ee-8d3d-4c56-a92d-296dd76101d7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.207381 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1005/2029] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:33.208007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905e0ee-8d3d-4c56-a92d-296dd76101d7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.208165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d84b2-3759-4a75-a53f-f0d9205afdd5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.214288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bfbb5-5083-4057-b027-3ff0b7cb5016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.218721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bfbb5-5083-4057-b027-3ff0b7cb5016 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.218721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bfbb5-5083-4057-b027-3ff0b7cb5016 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.218721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bfbb5-5083-4057-b027-3ff0b7cb5016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.220345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905e0ee-8d3d-4c56-a92d-296dd76101d7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.225156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffee83-d3c5-4038-a5d8-c2744d662680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.225806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffee83-d3c5-4038-a5d8-c2744d662680 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.226121 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffee83-d3c5-4038-a5d8-c2744d662680 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.226400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffee83-d3c5-4038-a5d8-c2744d662680 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.227131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffee83-d3c5-4038-a5d8-c2744d662680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.229296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffee83-d3c5-4038-a5d8-c2744d662680 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.233577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bfbb5-5083-4057-b027-3ff0b7cb5016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.235161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bfbb5-5083-4057-b027-3ff0b7cb5016 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.239482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1006/2030] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:33.240262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffee83-d3c5-4038-a5d8-c2744d662680 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.245664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1025/2031] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:33.246761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bfbb5-5083-4057-b027-3ff0b7cb5016 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.250362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffee83-d3c5-4038-a5d8-c2744d662680 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.256061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a59d65-7b9d-4ebd-8f0d-d046255df749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.256946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a59d65-7b9d-4ebd-8f0d-d046255df749 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.257227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a59d65-7b9d-4ebd-8f0d-d046255df749 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.257585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a59d65-7b9d-4ebd-8f0d-d046255df749 admin admin] PATH_INFO: `/v3/projects/53a8fc23c1d0449abe9eafe0dfa86c9f/users/80069db894b74c48a58afa1a0a4c209a/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.258304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bfbb5-5083-4057-b027-3ff0b7cb5016 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.263271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f4b13-6302-4883-b037-87b047c76b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.264827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f4b13-6302-4883-b037-87b047c76b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.264827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f4b13-6302-4883-b037-87b047c76b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.264827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f4b13-6302-4883-b037-87b047c76b35 admin admin] PATH_INFO: `/v3/projects/69eb3f4b63754bff8db43bf4e1228cb6/users/e9c4905f05ea42c689235aa995f097ab/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.275885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a59d65-7b9d-4ebd-8f0d-d046255df749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a8fc23c1d0449abe9eafe0dfa86c9f, user_id=80069db894b74c48a58afa1a0a4c209a, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.278569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a59d65-7b9d-4ebd-8f0d-d046255df749 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.281627 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f4b13-6302-4883-b037-87b047c76b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69eb3f4b63754bff8db43bf4e1228cb6, user_id=e9c4905f05ea42c689235aa995f097ab, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.284569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f4b13-6302-4883-b037-87b047c76b35 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.379830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1007/2032] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:33 2022] PUT /identity/v3/projects/53a8fc23c1d0449abe9eafe0dfa86c9f/users/80069db894b74c48a58afa1a0a4c209a/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:33.379933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1026/2033] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:33 2022] PUT /identity/v3/projects/69eb3f4b63754bff8db43bf4e1228cb6/users/e9c4905f05ea42c689235aa995f097ab/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:33.384391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a59d65-7b9d-4ebd-8f0d-d046255df749 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.391296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6d4cf-5ee7-4fcd-a534-8bb4bdec1116 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.391592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6d4cf-5ee7-4fcd-a534-8bb4bdec1116 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.391871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6d4cf-5ee7-4fcd-a534-8bb4bdec1116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.419435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b7a6d4cf-5ee7-4fcd-a534-8bb4bdec1116 None None] MFA Rules not processed for user `e9c4905f05ea42c689235aa995f097ab`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:33.420805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b7a6d4cf-5ee7-4fcd-a534-8bb4bdec1116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:33.452976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a59d65-7b9d-4ebd-8f0d-d046255df749 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.485458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1027/2034] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:33.489956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6d4cf-5ee7-4fcd-a534-8bb4bdec1116 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.519387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5406851-aedb-4ea3-a787-516e47aa4c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.520070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5406851-aedb-4ea3-a787-516e47aa4c3e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.521003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5406851-aedb-4ea3-a787-516e47aa4c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.521373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5406851-aedb-4ea3-a787-516e47aa4c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.522057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5406851-aedb-4ea3-a787-516e47aa4c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.523677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5406851-aedb-4ea3-a787-516e47aa4c3e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.535529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1008/2035] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:33.536350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5406851-aedb-4ea3-a787-516e47aa4c3e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.555546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6d4cf-5ee7-4fcd-a534-8bb4bdec1116 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.560215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f16c50-6638-4a11-b3ee-6da9ca596575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.560875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f16c50-6638-4a11-b3ee-6da9ca596575 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.561158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f16c50-6638-4a11-b3ee-6da9ca596575 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.561435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f16c50-6638-4a11-b3ee-6da9ca596575 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.562171 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f16c50-6638-4a11-b3ee-6da9ca596575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.564515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f16c50-6638-4a11-b3ee-6da9ca596575 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.595117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5406851-aedb-4ea3-a787-516e47aa4c3e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.598832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191b059-9523-4460-9972-133b337975a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.599477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191b059-9523-4460-9972-133b337975a0 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.599761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191b059-9523-4460-9972-133b337975a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.600039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191b059-9523-4460-9972-133b337975a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.656029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191b059-9523-4460-9972-133b337975a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.657516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191b059-9523-4460-9972-133b337975a0 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.667757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1009/2036] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:33.668501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191b059-9523-4460-9972-133b337975a0 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.682635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1028/2037] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:33 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:33.683228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191b059-9523-4460-9972-133b337975a0 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.683830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f16c50-6638-4a11-b3ee-6da9ca596575 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.747128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f16c50-6638-4a11-b3ee-6da9ca596575 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.749921 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18b77f-d7fd-4717-a703-8229997952e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.750802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18b77f-d7fd-4717-a703-8229997952e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.751088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18b77f-d7fd-4717-a703-8229997952e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.751369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18b77f-d7fd-4717-a703-8229997952e1 admin admin] PATH_INFO: `/v3/projects/53a8fc23c1d0449abe9eafe0dfa86c9f/users/80069db894b74c48a58afa1a0a4c209a/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.752474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b922b-49c1-4148-9cf7-0beedbe164a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.753143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b922b-49c1-4148-9cf7-0beedbe164a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.753427 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b922b-49c1-4148-9cf7-0beedbe164a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.753815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b922b-49c1-4148-9cf7-0beedbe164a4 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.754395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18b77f-d7fd-4717-a703-8229997952e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a8fc23c1d0449abe9eafe0dfa86c9f, user_id=80069db894b74c48a58afa1a0a4c209a, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.756302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b922b-49c1-4148-9cf7-0beedbe164a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.756486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b922b-49c1-4148-9cf7-0beedbe164a4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.757654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18b77f-d7fd-4717-a703-8229997952e1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.770817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1010/2038] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:33 2022] PUT /identity/v3/projects/53a8fc23c1d0449abe9eafe0dfa86c9f/users/80069db894b74c48a58afa1a0a4c209a/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:33.787418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f677ad-cbb4-4587-a535-c72d9ee09981 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.787736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f677ad-cbb4-4587-a535-c72d9ee09981 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.788015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f677ad-cbb4-4587-a535-c72d9ee09981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.816255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f6f677ad-cbb4-4587-a535-c72d9ee09981 None None] MFA Rules not processed for user `80069db894b74c48a58afa1a0a4c209a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:33.817570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f6f677ad-cbb4-4587-a535-c72d9ee09981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:33.847711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1029/2039] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:33 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:33.849039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b922b-49c1-4148-9cf7-0beedbe164a4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.858890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:33.861577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b922b-49c1-4148-9cf7-0beedbe164a4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.902033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1011/2040] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:33.926552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e286015-a1f9-40d1-80ec-2946ead540a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.927153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e286015-a1f9-40d1-80ec-2946ead540a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.927468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e286015-a1f9-40d1-80ec-2946ead540a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.927752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e286015-a1f9-40d1-80ec-2946ead540a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.928486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e286015-a1f9-40d1-80ec-2946ead540a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.930404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e286015-a1f9-40d1-80ec-2946ead540a9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:33.941698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1030/2041] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:33.945822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f677ad-cbb4-4587-a535-c72d9ee09981 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:33.958778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f677ad-cbb4-4587-a535-c72d9ee09981 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:33.964277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c459a-71e9-4c50-b64c-708ba6004eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:33.965178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c459a-71e9-4c50-b64c-708ba6004eea admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:33.965553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c459a-71e9-4c50-b64c-708ba6004eea admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:33.965837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c459a-71e9-4c50-b64c-708ba6004eea admin admin] PATH_INFO: `/v3/projects/24c16dbfe7fc44bc9b8af9e03cbeb918/users/83cd98e327b8492c95a51451487aa117/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:33.980786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c459a-71e9-4c50-b64c-708ba6004eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c16dbfe7fc44bc9b8af9e03cbeb918, user_id=83cd98e327b8492c95a51451487aa117, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:33.983764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c459a-71e9-4c50-b64c-708ba6004eea admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:34.099233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1012/2042] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:33 2022] PUT /identity/v3/projects/24c16dbfe7fc44bc9b8af9e03cbeb918/users/83cd98e327b8492c95a51451487aa117/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:34.103344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e286015-a1f9-40d1-80ec-2946ead540a9 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:34.113858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e286015-a1f9-40d1-80ec-2946ead540a9 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:34.140081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c459a-71e9-4c50-b64c-708ba6004eea admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:34.180154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53789c9b-ad07-4653-b75a-0363bea7b315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:34.180154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53789c9b-ad07-4653-b75a-0363bea7b315 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.180483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53789c9b-ad07-4653-b75a-0363bea7b315 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.180712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53789c9b-ad07-4653-b75a-0363bea7b315 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.182010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53789c9b-ad07-4653-b75a-0363bea7b315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:34.183628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53789c9b-ad07-4653-b75a-0363bea7b315 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:34.197641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1031/2043] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:34.198754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53789c9b-ad07-4653-b75a-0363bea7b315 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:34.199846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c459a-71e9-4c50-b64c-708ba6004eea admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:34.203695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050680-c3d4-4e79-8e18-73089976d941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:34.208905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050680-c3d4-4e79-8e18-73089976d941 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.208905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050680-c3d4-4e79-8e18-73089976d941 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.208905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050680-c3d4-4e79-8e18-73089976d941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.209237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53789c9b-ad07-4653-b75a-0363bea7b315 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:34.213136 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653f6cc-e2a6-4b78-8918-7e219aa1e1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:34.214057 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653f6cc-e2a6-4b78-8918-7e219aa1e1e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.214336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653f6cc-e2a6-4b78-8918-7e219aa1e1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.214611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653f6cc-e2a6-4b78-8918-7e219aa1e1e7 admin admin] PATH_INFO: `/v3/projects/24c16dbfe7fc44bc9b8af9e03cbeb918/users/83cd98e327b8492c95a51451487aa117/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.216670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653f6cc-e2a6-4b78-8918-7e219aa1e1e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c16dbfe7fc44bc9b8af9e03cbeb918, user_id=83cd98e327b8492c95a51451487aa117, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:34.219390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653f6cc-e2a6-4b78-8918-7e219aa1e1e7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:34.232173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1032/2044] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:34 2022] PUT /identity/v3/projects/24c16dbfe7fc44bc9b8af9e03cbeb918/users/83cd98e327b8492c95a51451487aa117/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:34.242554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a57094-0c6a-4a0b-8514-3b1a70c74a3d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.243122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a57094-0c6a-4a0b-8514-3b1a70c74a3d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.243407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a57094-0c6a-4a0b-8514-3b1a70c74a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.267480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0050680-c3d4-4e79-8e18-73089976d941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:34.268971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0050680-c3d4-4e79-8e18-73089976d941 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:34.270723 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-78a57094-0c6a-4a0b-8514-3b1a70c74a3d None None] MFA Rules not processed for user `83cd98e327b8492c95a51451487aa117`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:34.271994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-78a57094-0c6a-4a0b-8514-3b1a70c74a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:34.281206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1013/2045] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:34.346220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1033/2046] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:34.371050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050680-c3d4-4e79-8e18-73089976d941 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:34.375475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a57094-0c6a-4a0b-8514-3b1a70c74a3d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:34.381725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050680-c3d4-4e79-8e18-73089976d941 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:34.428038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a57094-0c6a-4a0b-8514-3b1a70c74a3d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:34.432445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646f0d3-4608-4b25-9987-428072145d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:34.433032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646f0d3-4608-4b25-9987-428072145d11 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.433315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646f0d3-4608-4b25-9987-428072145d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.433594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646f0d3-4608-4b25-9987-428072145d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.438386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:34.439259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.439546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.439823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] PATH_INFO: `/v3/users/9b551f2e38d84db6985ea75d32d7acf2` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.441075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b551f2e38d84db6985ea75d32d7acf2)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:34.443050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:34.443176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646f0d3-4608-4b25-9987-428072145d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:34.444593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646f0d3-4608-4b25-9987-428072145d11 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:34.457336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1034/2047] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:34.470521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:34.551746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646f0d3-4608-4b25-9987-428072145d11 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:34.560487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646f0d3-4608-4b25-9987-428072145d11 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:34.565924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c1240-fc94-47db-8c1e-698612cea7ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:34.565924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c1240-fc94-47db-8c1e-698612cea7ce service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.566119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c1240-fc94-47db-8c1e-698612cea7ce service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.566221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c1240-fc94-47db-8c1e-698612cea7ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.624516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c1240-fc94-47db-8c1e-698612cea7ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:34.625996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c1240-fc94-47db-8c1e-698612cea7ce service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:34.632734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b551f2e38d84db6985ea75d32d7acf2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:34.633642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b551f2e38d84db6985ea75d32d7acf2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:34.637487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1035/2048] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:34.655915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b551f2e38d84db6985ea75d32d7acf2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:34.667812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b551f2e38d84db6985ea75d32d7acf2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:34.679563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c1240-fc94-47db-8c1e-698612cea7ce service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:34.688592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c1240-fc94-47db-8c1e-698612cea7ce service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:34.692334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:34.692954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.693232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.693509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.695303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:34.736386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-72f2058f-ecbd-4d98-a9a8-07a34fbf5abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b551f2e38d84db6985ea75d32d7acf2'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:34.742871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1014/2049] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:34 2022] DELETE /identity/v3/users/9b551f2e38d84db6985ea75d32d7acf2 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:34.747133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7622e4-9754-4f12-a0f8-13b4835250dc None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.747421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7622e4-9754-4f12-a0f8-13b4835250dc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.747703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7622e4-9754-4f12-a0f8-13b4835250dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.763873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:34.764800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fe7622e4-9754-4f12-a0f8-13b4835250dc None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:34.765504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:34.766061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fe7622e4-9754-4f12-a0f8-13b4835250dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:34.778646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1036/2050] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:34.829253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1015/2051] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:34.839746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:34.841258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:34.898125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975bdce-3299-4476-91a9-84893fc2386e service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:34.945998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3c26-6f49-4b0f-a3ac-472953e078d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:34.946572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3c26-6f49-4b0f-a3ac-472953e078d7 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:34.946870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3c26-6f49-4b0f-a3ac-472953e078d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:34.947718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3c26-6f49-4b0f-a3ac-472953e078d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:34.956952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3c26-6f49-4b0f-a3ac-472953e078d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:34.958557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa3c26-6f49-4b0f-a3ac-472953e078d7 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:34.968319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1037/2052] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:35.104477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:35.190775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:35.597701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7622e4-9754-4f12-a0f8-13b4835250dc None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:35.599209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fe7622e4-9754-4f12-a0f8-13b4835250dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:35.611784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7622e4-9754-4f12-a0f8-13b4835250dc None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:35.616555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:35.617473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:35.617765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:35.618083 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] PATH_INFO: `/v3/projects/91de1e358d6b4adbafa953bbbd3d5e52` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:35.619390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=91de1e358d6b4adbafa953bbbd3d5e52)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:35.621549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:35.762446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91de1e358d6b4adbafa953bbbd3d5e52', 'request_id': 'req-f3b19b67-e145-4167-9a05-ac553d557e8f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:35.768085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91de1e358d6b4adbafa953bbbd3d5e52', 'request_id': 'req-f3b19b67-e145-4167-9a05-ac553d557e8f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:35.858822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91de1e358d6b4adbafa953bbbd3d5e52', 'request_id': 'req-f3b19b67-e145-4167-9a05-ac553d557e8f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:35.871647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] The token cache is being invalidate because project 91de1e358d6b4adbafa953bbbd3d5e52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:35.872148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:35.872854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:35.874432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1016/2053] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:35 2022] DELETE /identity/v3/projects/91de1e358d6b4adbafa953bbbd3d5e52 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:35.878389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3c26-6f49-4b0f-a3ac-472953e078d7 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:35.888510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3c26-6f49-4b0f-a3ac-472953e078d7 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:35.950234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:35.951112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:35.951401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:35.951689 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] PATH_INFO: `/v3/users/80069db894b74c48a58afa1a0a4c209a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:35.952982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] RBAC: Authorizing `identity:delete_user(user_id=80069db894b74c48a58afa1a0a4c209a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:35.955149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:36.218635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80069db894b74c48a58afa1a0a4c209a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:36.219284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80069db894b74c48a58afa1a0a4c209a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:36.236341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80069db894b74c48a58afa1a0a4c209a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:36.249899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80069db894b74c48a58afa1a0a4c209a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:36.419604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4b50b4ae-5ccf-4f42-86c0-0ede6834234b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80069db894b74c48a58afa1a0a4c209a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:36.433475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1038/2054] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:35 2022] DELETE /identity/v3/users/80069db894b74c48a58afa1a0a4c209a => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:36.773528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:36.894885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:36.896314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:36.911778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b19b67-e145-4167-9a05-ac553d557e8f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:36.971430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:36.972298 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:36.972590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:36.972900 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] PATH_INFO: `/v3/projects/53a8fc23c1d0449abe9eafe0dfa86c9f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:36.974112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a8fc23c1d0449abe9eafe0dfa86c9f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:36.976046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:37.036284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:37.080127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a8fc23c1d0449abe9eafe0dfa86c9f', 'request_id': 'req-c8838a7a-e586-4df8-949c-49f26d8efa5c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:37.085651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a8fc23c1d0449abe9eafe0dfa86c9f', 'request_id': 'req-c8838a7a-e586-4df8-949c-49f26d8efa5c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:37.188082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a8fc23c1d0449abe9eafe0dfa86c9f', 'request_id': 'req-c8838a7a-e586-4df8-949c-49f26d8efa5c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:37.200500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] The token cache is being invalidate because project 53a8fc23c1d0449abe9eafe0dfa86c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:37.200779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:37.201419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c8838a7a-e586-4df8-949c-49f26d8efa5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:37.203062 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1017/2055] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:36 2022] DELETE /identity/v3/projects/53a8fc23c1d0449abe9eafe0dfa86c9f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:37.207908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79eeb3-8b50-4522-99d0-4d613fc62cd3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:37.208356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79eeb3-8b50-4522-99d0-4d613fc62cd3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:37.208568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79eeb3-8b50-4522-99d0-4d613fc62cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:37.227271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4c79eeb3-8b50-4522-99d0-4d613fc62cd3 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:37.228573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4c79eeb3-8b50-4522-99d0-4d613fc62cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:37.285928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1039/2056] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:37.294661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645acb1-6d70-400d-8a5e-bb463da37e19 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:37.294948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645acb1-6d70-400d-8a5e-bb463da37e19 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:37.295224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645acb1-6d70-400d-8a5e-bb463da37e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:37.314441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f645acb1-6d70-400d-8a5e-bb463da37e19 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:37.315568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f645acb1-6d70-400d-8a5e-bb463da37e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:37.324835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1018/2057] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:37.329118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79eeb3-8b50-4522-99d0-4d613fc62cd3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:37.339758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79eeb3-8b50-4522-99d0-4d613fc62cd3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:37.344599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a23a66-df6d-412e-a4d7-f4e358740ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:37.345263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a23a66-df6d-412e-a4d7-f4e358740ccc admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:37.345552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a23a66-df6d-412e-a4d7-f4e358740ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:37.345836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a23a66-df6d-412e-a4d7-f4e358740ccc admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:37.346605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a23a66-df6d-412e-a4d7-f4e358740ccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:37.347006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a23a66-df6d-412e-a4d7-f4e358740ccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:37.348538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a23a66-df6d-412e-a4d7-f4e358740ccc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:37.361022 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1040/2058] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:59:37.365149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645acb1-6d70-400d-8a5e-bb463da37e19 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:37.375013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645acb1-6d70-400d-8a5e-bb463da37e19 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:37.380939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b951b-3435-4732-9e8d-39bd3575fb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:37.381599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b951b-3435-4732-9e8d-39bd3575fb06 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:37.381900 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b951b-3435-4732-9e8d-39bd3575fb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:37.382208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b951b-3435-4732-9e8d-39bd3575fb06 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:37.383078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b951b-3435-4732-9e8d-39bd3575fb06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:37.385430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b951b-3435-4732-9e8d-39bd3575fb06 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:37.600973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1019/2059] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:37 2022] POST /identity/v3/projects => generated 381 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:37.605619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a23a66-df6d-412e-a4d7-f4e358740ccc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:37.616130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a23a66-df6d-412e-a4d7-f4e358740ccc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:37.680787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45fe5c-565d-4c1e-be3a-675a5855d6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:37.681598 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45fe5c-565d-4c1e-be3a-675a5855d6da admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:37.681964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45fe5c-565d-4c1e-be3a-675a5855d6da admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:37.682198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45fe5c-565d-4c1e-be3a-675a5855d6da admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:37.683117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45fe5c-565d-4c1e-be3a-675a5855d6da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:37.685848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45fe5c-565d-4c1e-be3a-675a5855d6da admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:37.771556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1041/2060] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:37 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:37.773249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b951b-3435-4732-9e8d-39bd3575fb06 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:37.784550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b951b-3435-4732-9e8d-39bd3575fb06 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:37.790542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cead8c-6768-4d77-ae1d-db4b6ea8353d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:37.791332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cead8c-6768-4d77-ae1d-db4b6ea8353d admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:37.792580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cead8c-6768-4d77-ae1d-db4b6ea8353d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:37.792883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cead8c-6768-4d77-ae1d-db4b6ea8353d admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:37.793627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cead8c-6768-4d77-ae1d-db4b6ea8353d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:37.795477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cead8c-6768-4d77-ae1d-db4b6ea8353d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:37.807483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1020/2061] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:37.812157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45fe5c-565d-4c1e-be3a-675a5855d6da admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:37.823795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45fe5c-565d-4c1e-be3a-675a5855d6da admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:37.828679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb63de-fa8b-40ca-83e1-3d07171817fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:37.829324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb63de-fa8b-40ca-83e1-3d07171817fa admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:37.829649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb63de-fa8b-40ca-83e1-3d07171817fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:37.829957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb63de-fa8b-40ca-83e1-3d07171817fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:37.830725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb63de-fa8b-40ca-83e1-3d07171817fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:37.832324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb63de-fa8b-40ca-83e1-3d07171817fa admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:37.842082 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1042/2062] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:37.846117 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cead8c-6768-4d77-ae1d-db4b6ea8353d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:37.856073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cead8c-6768-4d77-ae1d-db4b6ea8353d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:37.860732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39502a84-a2cd-4b22-b807-c4626e99550d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:37.861634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39502a84-a2cd-4b22-b807-c4626e99550d admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:37.861938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39502a84-a2cd-4b22-b807-c4626e99550d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:37.867246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39502a84-a2cd-4b22-b807-c4626e99550d admin admin] PATH_INFO: `/v3/projects/3f570a74146f4b26b242194d2444c1a9/users/ba6726bc995f4774a95af3d38ada6ce4/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:37.877027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39502a84-a2cd-4b22-b807-c4626e99550d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f570a74146f4b26b242194d2444c1a9, user_id=ba6726bc995f4774a95af3d38ada6ce4, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:37.879760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39502a84-a2cd-4b22-b807-c4626e99550d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:37.997287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1021/2063] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:37 2022] PUT /identity/v3/projects/3f570a74146f4b26b242194d2444c1a9/users/ba6726bc995f4774a95af3d38ada6ce4/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:38.007987 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d689c-3d98-40b6-983c-fc0a9cf0eb57 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:38.008284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d689c-3d98-40b6-983c-fc0a9cf0eb57 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:38.008571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d689c-3d98-40b6-983c-fc0a9cf0eb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:38.047068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-170d689c-3d98-40b6-983c-fc0a9cf0eb57 None None] MFA Rules not processed for user `ba6726bc995f4774a95af3d38ada6ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:38.048436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-170d689c-3d98-40b6-983c-fc0a9cf0eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:38.152878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1043/2064] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:38.157392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39502a84-a2cd-4b22-b807-c4626e99550d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:38.225324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39502a84-a2cd-4b22-b807-c4626e99550d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:38.230913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ec42d-48de-44c4-bc59-615f87d583b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:38.231592 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ec42d-48de-44c4-bc59-615f87d583b9 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:38.231878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ec42d-48de-44c4-bc59-615f87d583b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:38.232157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ec42d-48de-44c4-bc59-615f87d583b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:38.232841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ec42d-48de-44c4-bc59-615f87d583b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:38.234874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ec42d-48de-44c4-bc59-615f87d583b9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:38.397391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1022/2065] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:38 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:38.401490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d689c-3d98-40b6-983c-fc0a9cf0eb57 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:38.412606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d689c-3d98-40b6-983c-fc0a9cf0eb57 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:38.469146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a3c86-4228-46c5-ac84-0d6ad9a65473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:38.469872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a3c86-4228-46c5-ac84-0d6ad9a65473 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:38.470204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a3c86-4228-46c5-ac84-0d6ad9a65473 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:38.470511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a3c86-4228-46c5-ac84-0d6ad9a65473 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:38.471222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a3c86-4228-46c5-ac84-0d6ad9a65473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:38.473146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a3c86-4228-46c5-ac84-0d6ad9a65473 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:38.578863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1044/2066] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:38 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:38.582788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ec42d-48de-44c4-bc59-615f87d583b9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:38.593675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ec42d-48de-44c4-bc59-615f87d583b9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:38.598863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfada71-5693-4893-8a12-e84a8ad5d07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:38.599544 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfada71-5693-4893-8a12-e84a8ad5d07f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:38.599852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfada71-5693-4893-8a12-e84a8ad5d07f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:38.600116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfada71-5693-4893-8a12-e84a8ad5d07f admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:38.600847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfada71-5693-4893-8a12-e84a8ad5d07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:38.602546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfada71-5693-4893-8a12-e84a8ad5d07f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:38.614122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1023/2067] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:38.618122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a3c86-4228-46c5-ac84-0d6ad9a65473 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:38.628148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a3c86-4228-46c5-ac84-0d6ad9a65473 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:38.632813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df601e-7390-46f0-b13b-9272949b3d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:38.633686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df601e-7390-46f0-b13b-9272949b3d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:38.634005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df601e-7390-46f0-b13b-9272949b3d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:38.634289 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df601e-7390-46f0-b13b-9272949b3d71 admin admin] PATH_INFO: `/v3/projects/b296718e35634e6ebd0742a1921d0110/users/8c949d36baeb4f45964055067c9df56b/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:38.649224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df601e-7390-46f0-b13b-9272949b3d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b296718e35634e6ebd0742a1921d0110, user_id=8c949d36baeb4f45964055067c9df56b, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:38.652488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df601e-7390-46f0-b13b-9272949b3d71 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:38.778202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1045/2068] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:38 2022] PUT /identity/v3/projects/b296718e35634e6ebd0742a1921d0110/users/8c949d36baeb4f45964055067c9df56b/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:38.782142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfada71-5693-4893-8a12-e84a8ad5d07f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:38.793061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfada71-5693-4893-8a12-e84a8ad5d07f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:38.857492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf619a0-6c2d-4765-98c9-50fac23e49db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:38.858306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf619a0-6c2d-4765-98c9-50fac23e49db admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:38.858669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf619a0-6c2d-4765-98c9-50fac23e49db admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:38.859023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf619a0-6c2d-4765-98c9-50fac23e49db admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:38.859887 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf619a0-6c2d-4765-98c9-50fac23e49db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:38.861764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf619a0-6c2d-4765-98c9-50fac23e49db admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:38.877122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df601e-7390-46f0-b13b-9272949b3d71 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:38.882058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1024/2069] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:38.888101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df601e-7390-46f0-b13b-9272949b3d71 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:38.892835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb673135-f5b0-4a13-a9d4-a81b30b3507d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:38.893727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb673135-f5b0-4a13-a9d4-a81b30b3507d admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:38.894047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb673135-f5b0-4a13-a9d4-a81b30b3507d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:38.894673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb673135-f5b0-4a13-a9d4-a81b30b3507d admin admin] PATH_INFO: `/v3/projects/b296718e35634e6ebd0742a1921d0110/users/8c949d36baeb4f45964055067c9df56b/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:38.896357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb673135-f5b0-4a13-a9d4-a81b30b3507d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b296718e35634e6ebd0742a1921d0110, user_id=8c949d36baeb4f45964055067c9df56b, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:38.899038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb673135-f5b0-4a13-a9d4-a81b30b3507d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:38.910669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1046/2070] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:38 2022] PUT /identity/v3/projects/b296718e35634e6ebd0742a1921d0110/users/8c949d36baeb4f45964055067c9df56b/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:38.921844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece76e4-c105-492d-a0a7-dd6faa73d30c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:38.922266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece76e4-c105-492d-a0a7-dd6faa73d30c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:38.922549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece76e4-c105-492d-a0a7-dd6faa73d30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:38.956213 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4ece76e4-c105-492d-a0a7-dd6faa73d30c None None] MFA Rules not processed for user `8c949d36baeb4f45964055067c9df56b`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:38.957410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4ece76e4-c105-492d-a0a7-dd6faa73d30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:39.022821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1025/2071] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:39.035652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb673135-f5b0-4a13-a9d4-a81b30b3507d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:39.037574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bb673135-f5b0-4a13-a9d4-a81b30b3507d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:39.095198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb673135-f5b0-4a13-a9d4-a81b30b3507d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:39.098855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a516e7-90d9-479b-95e7-fd0822304ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:39.099497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a516e7-90d9-479b-95e7-fd0822304ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:39.099783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a516e7-90d9-479b-95e7-fd0822304ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:39.100062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a516e7-90d9-479b-95e7-fd0822304ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:39.109054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a516e7-90d9-479b-95e7-fd0822304ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:39.110517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a516e7-90d9-479b-95e7-fd0822304ec5 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:39.120059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1047/2072] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:39.211124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:39.399782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:40.240304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece76e4-c105-492d-a0a7-dd6faa73d30c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:40.249951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece76e4-c105-492d-a0a7-dd6faa73d30c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:40.254754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8800f56-dd9b-42c1-9519-bd4cbfee06d7 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c949d36baeb4f45964055067c9df56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b296718e35634e6ebd0742a1921d0110', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:40.255417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8800f56-dd9b-42c1-9519-bd4cbfee06d7 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:40.255708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8800f56-dd9b-42c1-9519-bd4cbfee06d7 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:40.255993 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8800f56-dd9b-42c1-9519-bd4cbfee06d7 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:40.256724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8800f56-dd9b-42c1-9519-bd4cbfee06d7 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:40.257034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8800f56-dd9b-42c1-9519-bd4cbfee06d7 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:40.258719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8800f56-dd9b-42c1-9519-bd4cbfee06d7 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:40.270526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1026/2073] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:40.278782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a516e7-90d9-479b-95e7-fd0822304ec5 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:40.287848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a516e7-90d9-479b-95e7-fd0822304ec5 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:40.293108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a287c-fc6e-4dac-a299-cf1203b2d4b5 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c949d36baeb4f45964055067c9df56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b296718e35634e6ebd0742a1921d0110', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:40.293749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a287c-fc6e-4dac-a299-cf1203b2d4b5 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:40.294067 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a287c-fc6e-4dac-a299-cf1203b2d4b5 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:40.294351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a287c-fc6e-4dac-a299-cf1203b2d4b5 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:40.295039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a287c-fc6e-4dac-a299-cf1203b2d4b5 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:40.296910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a287c-fc6e-4dac-a299-cf1203b2d4b5 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:40.500855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1048/2074] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:40 2022] POST /identity/v3/projects => generated 354 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:40.504602 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8800f56-dd9b-42c1-9519-bd4cbfee06d7 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:40.514697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8800f56-dd9b-42c1-9519-bd4cbfee06d7 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:40.568564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c949d36baeb4f45964055067c9df56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b296718e35634e6ebd0742a1921d0110', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:40.569221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:40.569504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:40.569787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:40.570531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:40.570838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:40.572460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:40.583376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1027/2075] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:40.622231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a287c-fc6e-4dac-a299-cf1203b2d4b5 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:40.632322 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a287c-fc6e-4dac-a299-cf1203b2d4b5 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:40.637118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c949d36baeb4f45964055067c9df56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b296718e35634e6ebd0742a1921d0110', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:40.638011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:40.638295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:40.638568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] PATH_INFO: `/v3/projects/b2d74fa806fc40ddaed93d74cb725e16` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:40.639738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b2d74fa806fc40ddaed93d74cb725e16)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:40.641617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:40.767657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d74fa806fc40ddaed93d74cb725e16', 'request_id': 'req-1a88cb83-18b4-4d1d-b708-afef843f9e7f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:40.773033 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d74fa806fc40ddaed93d74cb725e16', 'request_id': 'req-1a88cb83-18b4-4d1d-b708-afef843f9e7f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:40.869150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d74fa806fc40ddaed93d74cb725e16', 'request_id': 'req-1a88cb83-18b4-4d1d-b708-afef843f9e7f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:40.882610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] The token cache is being invalidate because project b2d74fa806fc40ddaed93d74cb725e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:40.882922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:40.883563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:40.885333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1049/2076] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:40 2022] DELETE /identity/v3/projects/b2d74fa806fc40ddaed93d74cb725e16 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:41.122160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:41.328164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:41.621715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:41.623479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:41.688894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580e71-c0db-404e-957c-6719de876a53 tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:41.748040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:41.748912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:41.749197 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:41.749473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] PATH_INFO: `/v3/users/ba6726bc995f4774a95af3d38ada6ce4` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:41.750692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba6726bc995f4774a95af3d38ada6ce4)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:41.752663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:41.972753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba6726bc995f4774a95af3d38ada6ce4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:41.973568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba6726bc995f4774a95af3d38ada6ce4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:41.985536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba6726bc995f4774a95af3d38ada6ce4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:41.992226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba6726bc995f4774a95af3d38ada6ce4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.042546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:42.043963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:42.057004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88cb83-18b4-4d1d-b708-afef843f9e7f tempest-VolumeQuotaClassesTest-1570994339 tempest-VolumeQuotaClassesTest-1570994339-project-admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:42.061502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:42.062404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:42.062688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:42.062972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] PATH_INFO: `/v3/users/9c6e4c7f415643429e54f1d1897b6b47` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:42.067448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c6e4c7f415643429e54f1d1897b6b47)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:42.067448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:42.092918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-34b31e26-f6d2-4bee-9db3-5b7c140e2785 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba6726bc995f4774a95af3d38ada6ce4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.099433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1028/2077] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:41 2022] DELETE /identity/v3/users/ba6726bc995f4774a95af3d38ada6ce4 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:42.103903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9773e-7e91-451e-9175-fdf57524500b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:42.104181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9773e-7e91-451e-9175-fdf57524500b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:42.104462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9773e-7e91-451e-9175-fdf57524500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:42.121576 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6aa9773e-7e91-451e-9175-fdf57524500b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:42.123508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6aa9773e-7e91-451e-9175-fdf57524500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:42.181083 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1029/2078] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:42.190640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9773e-7e91-451e-9175-fdf57524500b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:42.191914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6aa9773e-7e91-451e-9175-fdf57524500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:42.247402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9773e-7e91-451e-9175-fdf57524500b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:42.251181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04152b5d-97a8-4b83-86d3-3716fa7474cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:42.251906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152b5d-97a8-4b83-86d3-3716fa7474cf service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:42.252189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152b5d-97a8-4b83-86d3-3716fa7474cf service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:42.252464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152b5d-97a8-4b83-86d3-3716fa7474cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:42.262931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04152b5d-97a8-4b83-86d3-3716fa7474cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:42.264409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04152b5d-97a8-4b83-86d3-3716fa7474cf service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:42.274806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1030/2079] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:42.291724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c6e4c7f415643429e54f1d1897b6b47'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.291724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c6e4c7f415643429e54f1d1897b6b47'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.297061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c6e4c7f415643429e54f1d1897b6b47'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.316202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c6e4c7f415643429e54f1d1897b6b47'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.338894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:42.407381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c6e4c7f415643429e54f1d1897b6b47'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.414114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1050/2080] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:42 2022] DELETE /identity/v3/users/9c6e4c7f415643429e54f1d1897b6b47 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:42.418700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793fcb6-8e15-4094-a240-b39ebe5ff7e4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:42.419170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793fcb6-8e15-4094-a240-b39ebe5ff7e4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:42.419263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793fcb6-8e15-4094-a240-b39ebe5ff7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:42.442026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c793fcb6-8e15-4094-a240-b39ebe5ff7e4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:42.443508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c793fcb6-8e15-4094-a240-b39ebe5ff7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:42.506450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1031/2081] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:42.521859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:42.523338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:42.566540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793fcb6-8e15-4094-a240-b39ebe5ff7e4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:42.568333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c793fcb6-8e15-4094-a240-b39ebe5ff7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:42.581801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793fcb6-8e15-4094-a240-b39ebe5ff7e4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:42.586788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:42.587872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:42.588252 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:42.588641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] PATH_INFO: `/v3/users/99d62013c90b4e29ab3e64ae7c85e6c0` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:42.590018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d62013c90b4e29ab3e64ae7c85e6c0)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:42.592087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:42.597764 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d608581-c738-433e-a952-77e98e5e1624 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:42.604757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071ccd1-2bb1-425f-a287-a0e8fd0164f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:42.605649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071ccd1-2bb1-425f-a287-a0e8fd0164f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:42.606053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071ccd1-2bb1-425f-a287-a0e8fd0164f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:42.606402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071ccd1-2bb1-425f-a287-a0e8fd0164f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:42.619077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071ccd1-2bb1-425f-a287-a0e8fd0164f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:42.621159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071ccd1-2bb1-425f-a287-a0e8fd0164f8 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:42.632517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1051/2082] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:42.852903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d62013c90b4e29ab3e64ae7c85e6c0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.853548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d62013c90b4e29ab3e64ae7c85e6c0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.865983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d62013c90b4e29ab3e64ae7c85e6c0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.872924 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d62013c90b4e29ab3e64ae7c85e6c0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.933701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071ccd1-2bb1-425f-a287-a0e8fd0164f8 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:42.935202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b071ccd1-2bb1-425f-a287-a0e8fd0164f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:42.958073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071ccd1-2bb1-425f-a287-a0e8fd0164f8 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:42.960269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:42.960817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:42.961159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:42.961424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] PATH_INFO: `/v3/projects/3f570a74146f4b26b242194d2444c1a9` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:42.962676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f570a74146f4b26b242194d2444c1a9)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:42.965170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:42.980899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b33befd7-5a01-4a9d-b8ef-108e6a14ae6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d62013c90b4e29ab3e64ae7c85e6c0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:42.988050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1032/2083] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:42 2022] DELETE /identity/v3/users/99d62013c90b4e29ab3e64ae7c85e6c0 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:42.989761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a944cc-6e3b-4999-b2bf-9b910cfeb320 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:42.990189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a944cc-6e3b-4999-b2bf-9b910cfeb320 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:42.990968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a944cc-6e3b-4999-b2bf-9b910cfeb320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:43.015343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d5a944cc-6e3b-4999-b2bf-9b910cfeb320 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:43.015343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d5a944cc-6e3b-4999-b2bf-9b910cfeb320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:43.081369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1033/2084] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:43.090897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a944cc-6e3b-4999-b2bf-9b910cfeb320 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:43.092256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d5a944cc-6e3b-4999-b2bf-9b910cfeb320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:43.121458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f570a74146f4b26b242194d2444c1a9', 'request_id': 'req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.126958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f570a74146f4b26b242194d2444c1a9', 'request_id': 'req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.149451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a944cc-6e3b-4999-b2bf-9b910cfeb320 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:43.196390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f570a74146f4b26b242194d2444c1a9', 'request_id': 'req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.197286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:43.204841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57c227-6059-4053-b7b9-f3fe65752b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:43.205041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57c227-6059-4053-b7b9-f3fe65752b27 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:43.205312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57c227-6059-4053-b7b9-f3fe65752b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:43.205643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57c227-6059-4053-b7b9-f3fe65752b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:43.209939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] The token cache is being invalidate because project 3f570a74146f4b26b242194d2444c1a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:43.210231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.210865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.212587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1052/2085] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:42 2022] DELETE /identity/v3/projects/3f570a74146f4b26b242194d2444c1a9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:43.213516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:43.214842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:43.277356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57c227-6059-4053-b7b9-f3fe65752b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:43.279023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57c227-6059-4053-b7b9-f3fe65752b27 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:43.289349 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1034/2086] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:43.296467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57c227-6059-4053-b7b9-f3fe65752b27 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:43.296467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5a57c227-6059-4053-b7b9-f3fe65752b27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:43.297255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e235bf3-0b3d-4fab-b91d-37ae8f02e94c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:43.304270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:43.304839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57c227-6059-4053-b7b9-f3fe65752b27 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:43.305730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:43.306055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:43.306341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] PATH_INFO: `/v3/projects/d4249c6d236241ec892a304cc57d03e9` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:43.308438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4249c6d236241ec892a304cc57d03e9)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:43.310575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:43.311489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:43.312551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:43.313047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:43.313499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] PATH_INFO: `/v3/users/8c949d36baeb4f45964055067c9df56b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:43.314996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c949d36baeb4f45964055067c9df56b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:43.317376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:43.764398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c949d36baeb4f45964055067c9df56b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.765323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c949d36baeb4f45964055067c9df56b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.783286 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4249c6d236241ec892a304cc57d03e9', 'request_id': 'req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.783501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c949d36baeb4f45964055067c9df56b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.791236 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4249c6d236241ec892a304cc57d03e9', 'request_id': 'req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.799299 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c949d36baeb4f45964055067c9df56b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.858744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4249c6d236241ec892a304cc57d03e9', 'request_id': 'req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.871115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] The token cache is being invalidate because project d4249c6d236241ec892a304cc57d03e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:43.871493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.872262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.874201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1053/2087] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:43 2022] DELETE /identity/v3/projects/d4249c6d236241ec892a304cc57d03e9 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:43.877738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:43.879214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:43.964765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f933eea-5ca2-4a0a-a0b2-945b0c6bf720 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:43.969714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:43.970636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:43.970919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:43.971200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] PATH_INFO: `/v3/users/6d5e6cd4f3af4255b97dd0dcab625b59` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:43.972445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d5e6cd4f3af4255b97dd0dcab625b59)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:43.974484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:43.979373 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:43.985006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c949d36baeb4f45964055067c9df56b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:43.994133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1035/2088] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:43 2022] DELETE /identity/v3/users/8c949d36baeb4f45964055067c9df56b => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:43.999840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:43.999840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:44.072941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891be3f-f74b-42b8-846b-438cef2b046b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:44.078753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:44.086911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:44.086911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:44.086911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] PATH_INFO: `/v3/projects/dc646fd36ff140bc9e3cae95da69b1c0` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:44.086911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc646fd36ff140bc9e3cae95da69b1c0)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:44.086911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:44.206059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d5e6cd4f3af4255b97dd0dcab625b59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.207651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d5e6cd4f3af4255b97dd0dcab625b59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.209398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc646fd36ff140bc9e3cae95da69b1c0', 'request_id': 'req-17ff1a49-094d-40b1-8840-6f161d511415'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.215446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc646fd36ff140bc9e3cae95da69b1c0', 'request_id': 'req-17ff1a49-094d-40b1-8840-6f161d511415'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.216097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d5e6cd4f3af4255b97dd0dcab625b59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.228653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d5e6cd4f3af4255b97dd0dcab625b59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.321843 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc646fd36ff140bc9e3cae95da69b1c0', 'request_id': 'req-17ff1a49-094d-40b1-8840-6f161d511415'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.333893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] The token cache is being invalidate because project dc646fd36ff140bc9e3cae95da69b1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:44.334208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.334852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.336424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1036/2089] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:43 2022] DELETE /identity/v3/projects/dc646fd36ff140bc9e3cae95da69b1c0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:44.340215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:44.341649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:44.382018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d5e6cd4f3af4255b97dd0dcab625b59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.388952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1054/2090] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:43 2022] DELETE /identity/v3/users/6d5e6cd4f3af4255b97dd0dcab625b59 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:44.409811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff1a49-094d-40b1-8840-6f161d511415 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:44.429949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:44.434886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:44.480636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:44.481733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:44.482058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:44.484518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] PATH_INFO: `/v3/users/e9c4905f05ea42c689235aa995f097ab` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:44.484518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9c4905f05ea42c689235aa995f097ab)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:44.486055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:44.520075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfa159-e89a-498c-acdf-5a13279fbd19 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:44.525023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:44.525891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:44.526331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:44.527133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] PATH_INFO: `/v3/projects/b296718e35634e6ebd0742a1921d0110` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:44.528368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b296718e35634e6ebd0742a1921d0110)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:44.530321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:44.731612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b296718e35634e6ebd0742a1921d0110', 'request_id': 'req-eba85486-6a7b-4abb-b24d-08aea3f845c6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.736510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9c4905f05ea42c689235aa995f097ab'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.739993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b296718e35634e6ebd0742a1921d0110', 'request_id': 'req-eba85486-6a7b-4abb-b24d-08aea3f845c6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.740566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9c4905f05ea42c689235aa995f097ab'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.768145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9c4905f05ea42c689235aa995f097ab'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.776181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9c4905f05ea42c689235aa995f097ab'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.921544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b296718e35634e6ebd0742a1921d0110', 'request_id': 'req-eba85486-6a7b-4abb-b24d-08aea3f845c6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.932983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] The token cache is being invalidate because project b296718e35634e6ebd0742a1921d0110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:44.933267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.933908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-eba85486-6a7b-4abb-b24d-08aea3f845c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:44.935453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1055/2091] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:44 2022] DELETE /identity/v3/projects/b296718e35634e6ebd0742a1921d0110 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:44.940102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a0669-0e0e-4600-8695-52dd89e7c12d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:44.940384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a0669-0e0e-4600-8695-52dd89e7c12d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:44.940659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a0669-0e0e-4600-8695-52dd89e7c12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:44.959085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1e6a0669-0e0e-4600-8695-52dd89e7c12d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:44.960258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1e6a0669-0e0e-4600-8695-52dd89e7c12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:45.021407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1056/2092] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:45.025462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a0669-0e0e-4600-8695-52dd89e7c12d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:45.025462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1e6a0669-0e0e-4600-8695-52dd89e7c12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:45.036270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a0669-0e0e-4600-8695-52dd89e7c12d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:45.041769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5e8be610-afdd-4207-87f6-684e105f549f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9c4905f05ea42c689235aa995f097ab'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.046287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:45.056821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:45.056821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:45.056821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] PATH_INFO: `/v3/projects/f3bfadc5233c49278573688ea6f063da` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:45.056821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3bfadc5233c49278573688ea6f063da)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:45.057406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1037/2093] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:44 2022] DELETE /identity/v3/users/e9c4905f05ea42c689235aa995f097ab => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:45.057504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:45.057657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f185c6-428f-40c7-aeb5-997772050608 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:45.057657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f185c6-428f-40c7-aeb5-997772050608 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:45.057657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f185c6-428f-40c7-aeb5-997772050608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:45.086286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-49f185c6-428f-40c7-aeb5-997772050608 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:45.087763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-49f185c6-428f-40c7-aeb5-997772050608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:45.187131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1038/2094] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:45.192151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f185c6-428f-40c7-aeb5-997772050608 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:45.205218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f185c6-428f-40c7-aeb5-997772050608 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:45.222032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3bfadc5233c49278573688ea6f063da', 'request_id': 'req-9f48d8f8-cecd-443d-a22d-440132ecd53d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.228928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3bfadc5233c49278573688ea6f063da', 'request_id': 'req-9f48d8f8-cecd-443d-a22d-440132ecd53d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.277256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38833afe-a190-43ff-b9e0-11a3b0ccfe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:45.277963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38833afe-a190-43ff-b9e0-11a3b0ccfe1c admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:45.278252 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38833afe-a190-43ff-b9e0-11a3b0ccfe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:45.278560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38833afe-a190-43ff-b9e0-11a3b0ccfe1c admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:45.279330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38833afe-a190-43ff-b9e0-11a3b0ccfe1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:45.279649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38833afe-a190-43ff-b9e0-11a3b0ccfe1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:45.281359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38833afe-a190-43ff-b9e0-11a3b0ccfe1c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:45.293874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1039/2095] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:45.298341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38833afe-a190-43ff-b9e0-11a3b0ccfe1c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:45.313461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38833afe-a190-43ff-b9e0-11a3b0ccfe1c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:45.314099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09796c90-e69f-4121-9264-10e4132c4e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:45.314815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09796c90-e69f-4121-9264-10e4132c4e62 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:45.315244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09796c90-e69f-4121-9264-10e4132c4e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:45.315584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09796c90-e69f-4121-9264-10e4132c4e62 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:45.316289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09796c90-e69f-4121-9264-10e4132c4e62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:45.319227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09796c90-e69f-4121-9264-10e4132c4e62 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:45.332994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3bfadc5233c49278573688ea6f063da', 'request_id': 'req-9f48d8f8-cecd-443d-a22d-440132ecd53d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.349684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] The token cache is being invalidate because project f3bfadc5233c49278573688ea6f063da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:45.350157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.351006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.353247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1057/2096] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:45 2022] DELETE /identity/v3/projects/f3bfadc5233c49278573688ea6f063da => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:45.358655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:45.363657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:45.376017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48d8f8-cecd-443d-a22d-440132ecd53d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:45.380770 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:45.386061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:45.386061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:45.386061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] PATH_INFO: `/v3/users/c8432f7513774619b4cb3d4742cdff0a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:45.386061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8432f7513774619b4cb3d4742cdff0a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:45.386061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:45.477281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1040/2097] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:45 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:45.482142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09796c90-e69f-4121-9264-10e4132c4e62 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:45.483812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-09796c90-e69f-4121-9264-10e4132c4e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:45.545448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09796c90-e69f-4121-9264-10e4132c4e62 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:45.550056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93180289-201d-4b10-8239-7af27f03a932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:45.550715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93180289-201d-4b10-8239-7af27f03a932 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:45.550995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93180289-201d-4b10-8239-7af27f03a932 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:45.551274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93180289-201d-4b10-8239-7af27f03a932 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:45.551946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93180289-201d-4b10-8239-7af27f03a932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:45.553915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93180289-201d-4b10-8239-7af27f03a932 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:45.628295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:45.691003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1041/2098] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:45 2022] POST /identity/v3/users => generated 284 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:45.691899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93180289-201d-4b10-8239-7af27f03a932 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:45.694087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-93180289-201d-4b10-8239-7af27f03a932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:45.698254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8432f7513774619b4cb3d4742cdff0a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.699424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8432f7513774619b4cb3d4742cdff0a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.707323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8432f7513774619b4cb3d4742cdff0a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.716492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93180289-201d-4b10-8239-7af27f03a932 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:45.716492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:45.718827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:45.719863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:45.720303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:45.720595 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] PATH_INFO: `/v3/projects/69eb3f4b63754bff8db43bf4e1228cb6` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:45.721831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69eb3f4b63754bff8db43bf4e1228cb6)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:45.724086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8432f7513774619b4cb3d4742cdff0a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.724246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:45.799301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8432f7513774619b4cb3d4742cdff0a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.805952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1058/2099] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:45 2022] DELETE /identity/v3/users/c8432f7513774619b4cb3d4742cdff0a => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:45.806837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:45.808221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:45.879406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a3b05-aa8f-44d6-9c5e-7329738127a1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:45.887956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69eb3f4b63754bff8db43bf4e1228cb6', 'request_id': 'req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.895282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69eb3f4b63754bff8db43bf4e1228cb6', 'request_id': 'req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:45.936652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e69cc-dba8-446a-a05b-67adc6614459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:45.937318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e69cc-dba8-446a-a05b-67adc6614459 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:45.937601 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e69cc-dba8-446a-a05b-67adc6614459 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:45.937872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e69cc-dba8-446a-a05b-67adc6614459 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:45.938616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e69cc-dba8-446a-a05b-67adc6614459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:45.940261 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e69cc-dba8-446a-a05b-67adc6614459 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:45.950159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1059/2100] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:45.954012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e69cc-dba8-446a-a05b-67adc6614459 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:45.963204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e69cc-dba8-446a-a05b-67adc6614459 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:45.968231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c61a-96f5-497a-9f14-59c4d9641536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:45.968828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c61a-96f5-497a-9f14-59c4d9641536 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:45.969108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c61a-96f5-497a-9f14-59c4d9641536 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:45.969382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c61a-96f5-497a-9f14-59c4d9641536 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:45.970104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45c61a-96f5-497a-9f14-59c4d9641536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:45.971657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45c61a-96f5-497a-9f14-59c4d9641536 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:45.981424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1060/2101] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:45.988833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c61a-96f5-497a-9f14-59c4d9641536 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:45.998790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c61a-96f5-497a-9f14-59c4d9641536 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:46.003319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195b49-a137-47a4-ba75-b7b24f0a4d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:46.004143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195b49-a137-47a4-ba75-b7b24f0a4d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:46.004428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195b49-a137-47a4-ba75-b7b24f0a4d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:46.004712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195b49-a137-47a4-ba75-b7b24f0a4d96 admin admin] PATH_INFO: `/v3/projects/5e7e805c429e4b61a9530da191f327a2/users/6122c5b2981a4018a8c93dfa72b65919/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:46.006992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69eb3f4b63754bff8db43bf4e1228cb6', 'request_id': 'req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.019350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] The token cache is being invalidate because project 69eb3f4b63754bff8db43bf4e1228cb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:46.019597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195b49-a137-47a4-ba75-b7b24f0a4d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e7e805c429e4b61a9530da191f327a2, user_id=6122c5b2981a4018a8c93dfa72b65919, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:46.019740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.020397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.022014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1042/2102] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:45 2022] DELETE /identity/v3/projects/69eb3f4b63754bff8db43bf4e1228cb6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:46.022283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195b49-a137-47a4-ba75-b7b24f0a4d96 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:46.025795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:46.027757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:46.040315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d72bc3-03d9-4c5b-8a08-71fa1749f4b6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:46.045701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:46.049566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:46.049566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:46.049566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] PATH_INFO: `/v3/users/83cd98e327b8492c95a51451487aa117` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:46.049566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=83cd98e327b8492c95a51451487aa117)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:46.050345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:46.102902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1061/2103] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:45 2022] PUT /identity/v3/projects/5e7e805c429e4b61a9530da191f327a2/users/6122c5b2981a4018a8c93dfa72b65919/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:46.115848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb9baf-3cbb-4c50-933f-bc8eefa3297c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:46.116246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb9baf-3cbb-4c50-933f-bc8eefa3297c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:46.116621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb9baf-3cbb-4c50-933f-bc8eefa3297c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:46.143643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-69fb9baf-3cbb-4c50-933f-bc8eefa3297c None None] MFA Rules not processed for user `6122c5b2981a4018a8c93dfa72b65919`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:46.144929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-69fb9baf-3cbb-4c50-933f-bc8eefa3297c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:46.213428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1062/2104] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:46.226180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb9baf-3cbb-4c50-933f-bc8eefa3297c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:46.227634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-69fb9baf-3cbb-4c50-933f-bc8eefa3297c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:46.297665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb9baf-3cbb-4c50-933f-bc8eefa3297c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:46.304245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807371-6840-467c-9f35-af2c109c93fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:46.304245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807371-6840-467c-9f35-af2c109c93fc admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:46.304245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807371-6840-467c-9f35-af2c109c93fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:46.304245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807371-6840-467c-9f35-af2c109c93fc admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:46.304609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39807371-6840-467c-9f35-af2c109c93fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:46.306259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39807371-6840-467c-9f35-af2c109c93fc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:46.372926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83cd98e327b8492c95a51451487aa117'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.373586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83cd98e327b8492c95a51451487aa117'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.386754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83cd98e327b8492c95a51451487aa117'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.399099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83cd98e327b8492c95a51451487aa117'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.477910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1063/2105] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:46 2022] POST /identity/v3/projects => generated 365 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:46.477910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807371-6840-467c-9f35-af2c109c93fc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:46.477910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-39807371-6840-467c-9f35-af2c109c93fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:46.537939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807371-6840-467c-9f35-af2c109c93fc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:46.538670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83cd98e327b8492c95a51451487aa117'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.543122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:46.544020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:46.544306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:46.544633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] PATH_INFO: `/v3/projects/2840df1db5764778ae9a22ba3dea99b3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:46.545723 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1043/2106] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:46 2022] DELETE /identity/v3/users/83cd98e327b8492c95a51451487aa117 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:46.545919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2840df1db5764778ae9a22ba3dea99b3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:46.546522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:46.547823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:46.547946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:46.617187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4309b0-8030-4018-affe-3e5e4c938a8d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:46.623253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaea461-c3a1-41a6-a026-abbe83613370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:46.623253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaea461-c3a1-41a6-a026-abbe83613370 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:46.623253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaea461-c3a1-41a6-a026-abbe83613370 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:46.623253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaea461-c3a1-41a6-a026-abbe83613370 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:46.623776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaea461-c3a1-41a6-a026-abbe83613370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:46.625732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaea461-c3a1-41a6-a026-abbe83613370 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:46.658321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2840df1db5764778ae9a22ba3dea99b3', 'request_id': 'req-8984dd7a-3f92-460f-8190-71d3a9ba856d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.663604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2840df1db5764778ae9a22ba3dea99b3', 'request_id': 'req-8984dd7a-3f92-460f-8190-71d3a9ba856d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.701256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1044/2107] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:46 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:46.701256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaea461-c3a1-41a6-a026-abbe83613370 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:46.701256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dcaea461-c3a1-41a6-a026-abbe83613370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:46.712442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaea461-c3a1-41a6-a026-abbe83613370 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:46.768522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:46.769403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:46.769689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:46.769998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] PATH_INFO: `/v3/users/181022a415f7483d8ad1099231d4c5b1` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:46.771210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=181022a415f7483d8ad1099231d4c5b1)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:46.773242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:46.785455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2840df1db5764778ae9a22ba3dea99b3', 'request_id': 'req-8984dd7a-3f92-460f-8190-71d3a9ba856d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.799424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] The token cache is being invalidate because project 2840df1db5764778ae9a22ba3dea99b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:46.799847 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.800640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:46.802280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1064/2108] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:46 2022] DELETE /identity/v3/projects/2840df1db5764778ae9a22ba3dea99b3 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:46.803040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:46.804323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:46.824336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984dd7a-3f92-460f-8190-71d3a9ba856d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:46.824336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03bf7f-6544-4afd-a771-997d506f5587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:46.824336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03bf7f-6544-4afd-a771-997d506f5587 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:46.824336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03bf7f-6544-4afd-a771-997d506f5587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:46.824758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03bf7f-6544-4afd-a771-997d506f5587 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:46.825230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03bf7f-6544-4afd-a771-997d506f5587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:46.826959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03bf7f-6544-4afd-a771-997d506f5587 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:46.838364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1065/2109] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:46.843293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03bf7f-6544-4afd-a771-997d506f5587 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:46.853699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03bf7f-6544-4afd-a771-997d506f5587 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:46.861491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa77ee4-b751-4c70-8e32-542c58cf81f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:46.862453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa77ee4-b751-4c70-8e32-542c58cf81f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:46.863329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa77ee4-b751-4c70-8e32-542c58cf81f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:46.863636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa77ee4-b751-4c70-8e32-542c58cf81f0 admin admin] PATH_INFO: `/v3/projects/91444d95f649480ba256a4dd5799aa72/users/d9bbb5bf7b234109a00256d5ba6866cb/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:46.887881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa77ee4-b751-4c70-8e32-542c58cf81f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91444d95f649480ba256a4dd5799aa72, user_id=d9bbb5bf7b234109a00256d5ba6866cb, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:46.890938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa77ee4-b751-4c70-8e32-542c58cf81f0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:46.988597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:47.028511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181022a415f7483d8ad1099231d4c5b1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.029111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181022a415f7483d8ad1099231d4c5b1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.041160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181022a415f7483d8ad1099231d4c5b1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.051555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181022a415f7483d8ad1099231d4c5b1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.104538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1066/2110] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:46 2022] PUT /identity/v3/projects/91444d95f649480ba256a4dd5799aa72/users/d9bbb5bf7b234109a00256d5ba6866cb/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:47.108605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa77ee4-b751-4c70-8e32-542c58cf81f0 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:47.110333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5aa77ee4-b751-4c70-8e32-542c58cf81f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.155855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181022a415f7483d8ad1099231d4c5b1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.163580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1045/2111] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:46 2022] DELETE /identity/v3/users/181022a415f7483d8ad1099231d4c5b1 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:47.164424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:47.168000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.177729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa77ee4-b751-4c70-8e32-542c58cf81f0 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:47.236667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9320e-219d-458b-8389-f5d8f6d057cf admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:47.238451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45df50-8acc-4072-8fd5-0fe1471dd345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:47.239327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45df50-8acc-4072-8fd5-0fe1471dd345 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.239875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45df50-8acc-4072-8fd5-0fe1471dd345 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.240015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45df50-8acc-4072-8fd5-0fe1471dd345 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.240712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45df50-8acc-4072-8fd5-0fe1471dd345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:47.242584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45df50-8acc-4072-8fd5-0fe1471dd345 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:47.246481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:47.246481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.246481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.246481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] PATH_INFO: `/v3/projects/24c16dbfe7fc44bc9b8af9e03cbeb918` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.247393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c16dbfe7fc44bc9b8af9e03cbeb918)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:47.249416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:47.253591 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1067/2112] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:47.255043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab69aa1-8c54-4967-a7e8-97673c788dc9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.255340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab69aa1-8c54-4967-a7e8-97673c788dc9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.255621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab69aa1-8c54-4967-a7e8-97673c788dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.273193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7ab69aa1-8c54-4967-a7e8-97673c788dc9 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:47.274570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7ab69aa1-8c54-4967-a7e8-97673c788dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.282493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1068/2113] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:47.283210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab69aa1-8c54-4967-a7e8-97673c788dc9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:47.293126 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab69aa1-8c54-4967-a7e8-97673c788dc9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:47.297787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c25c2-4314-45d6-8b7d-727f4ed8bf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:47.299628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c25c2-4314-45d6-8b7d-727f4ed8bf66 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.299628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c25c2-4314-45d6-8b7d-727f4ed8bf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.299628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c25c2-4314-45d6-8b7d-727f4ed8bf66 admin admin] PATH_INFO: `/v3/projects/91444d95f649480ba256a4dd5799aa72/users/d9bbb5bf7b234109a00256d5ba6866cb/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.301224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c25c2-4314-45d6-8b7d-727f4ed8bf66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91444d95f649480ba256a4dd5799aa72, user_id=d9bbb5bf7b234109a00256d5ba6866cb, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:47.303913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c25c2-4314-45d6-8b7d-727f4ed8bf66 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:47.315065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1069/2114] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:47 2022] PUT /identity/v3/projects/91444d95f649480ba256a4dd5799aa72/users/d9bbb5bf7b234109a00256d5ba6866cb/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:47.315632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c25c2-4314-45d6-8b7d-727f4ed8bf66 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:47.317234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-983c25c2-4314-45d6-8b7d-727f4ed8bf66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.373848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c25c2-4314-45d6-8b7d-727f4ed8bf66 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:47.377719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8a4d8-1ea3-473e-b36e-53a32e57d12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:47.378475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8a4d8-1ea3-473e-b36e-53a32e57d12d service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.378761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8a4d8-1ea3-473e-b36e-53a32e57d12d service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.379045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8a4d8-1ea3-473e-b36e-53a32e57d12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.420448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c16dbfe7fc44bc9b8af9e03cbeb918', 'request_id': 'req-60756aa6-a121-47e7-b968-4c4b77f04167'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.427845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c16dbfe7fc44bc9b8af9e03cbeb918', 'request_id': 'req-60756aa6-a121-47e7-b968-4c4b77f04167'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.442807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8a4d8-1ea3-473e-b36e-53a32e57d12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:47.444244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8a4d8-1ea3-473e-b36e-53a32e57d12d service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:47.454176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1070/2115] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:47.455570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a620ca-ab0c-4bf1-a215-3c52f6f39e11 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.455864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a620ca-ab0c-4bf1-a215-3c52f6f39e11 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.456144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a620ca-ab0c-4bf1-a215-3c52f6f39e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.482819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d0a620ca-ab0c-4bf1-a215-3c52f6f39e11 None None] MFA Rules not processed for user `d9bbb5bf7b234109a00256d5ba6866cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:47.484028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d0a620ca-ab0c-4bf1-a215-3c52f6f39e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.531291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c16dbfe7fc44bc9b8af9e03cbeb918', 'request_id': 'req-60756aa6-a121-47e7-b968-4c4b77f04167'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.545562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] The token cache is being invalidate because project 24c16dbfe7fc44bc9b8af9e03cbeb918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:47.546336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.546561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60756aa6-a121-47e7-b968-4c4b77f04167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:47.554433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1046/2116] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:47 2022] DELETE /identity/v3/projects/24c16dbfe7fc44bc9b8af9e03cbeb918 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:47.559111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb584ebf-0374-4597-9d5a-70f81f646821 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.559111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb584ebf-0374-4597-9d5a-70f81f646821 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.559111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb584ebf-0374-4597-9d5a-70f81f646821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.563764 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1071/2117] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:47.567223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a620ca-ab0c-4bf1-a215-3c52f6f39e11 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:47.568679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d0a620ca-ab0c-4bf1-a215-3c52f6f39e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.583857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bb584ebf-0374-4597-9d5a-70f81f646821 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:47.585327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bb584ebf-0374-4597-9d5a-70f81f646821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.618788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:47.628551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a620ca-ab0c-4bf1-a215-3c52f6f39e11 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:47.632951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098d253-e9d1-4e8d-90d9-d703cc8d5b36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:47.633780 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098d253-e9d1-4e8d-90d9-d703cc8d5b36 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.634075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098d253-e9d1-4e8d-90d9-d703cc8d5b36 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.634679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098d253-e9d1-4e8d-90d9-d703cc8d5b36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.636326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a098d253-e9d1-4e8d-90d9-d703cc8d5b36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.648850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098d253-e9d1-4e8d-90d9-d703cc8d5b36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:47.654728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1047/2118] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:47.654870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098d253-e9d1-4e8d-90d9-d703cc8d5b36 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:47.659397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aabc0-9c51-4e03-b237-8f7888944103 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.659698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aabc0-9c51-4e03-b237-8f7888944103 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.662890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aabc0-9c51-4e03-b237-8f7888944103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.662988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1072/2119] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:47.683638 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a02aabc0-9c51-4e03-b237-8f7888944103 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:47.684933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a02aabc0-9c51-4e03-b237-8f7888944103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.694174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1048/2120] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:47.698734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098d253-e9d1-4e8d-90d9-d703cc8d5b36 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:47.708999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098d253-e9d1-4e8d-90d9-d703cc8d5b36 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:47.713674 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bc493-fbf2-410a-b64d-3d5356f7a1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:47.714915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bc493-fbf2-410a-b64d-3d5356f7a1db admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.715157 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bc493-fbf2-410a-b64d-3d5356f7a1db admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.715481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bc493-fbf2-410a-b64d-3d5356f7a1db admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.716150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bc493-fbf2-410a-b64d-3d5356f7a1db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:47.716453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bc493-fbf2-410a-b64d-3d5356f7a1db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:47.718093 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bc493-fbf2-410a-b64d-3d5356f7a1db admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:47.731186 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1073/2121] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 09:59:47.735292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aabc0-9c51-4e03-b237-8f7888944103 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:47.744671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aabc0-9c51-4e03-b237-8f7888944103 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:47.750837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0cdb3-184b-4441-b732-9091d7e03b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:47.751506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0cdb3-184b-4441-b732-9091d7e03b92 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.751795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0cdb3-184b-4441-b732-9091d7e03b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.752063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0cdb3-184b-4441-b732-9091d7e03b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.752754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0cdb3-184b-4441-b732-9091d7e03b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:47.754910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0cdb3-184b-4441-b732-9091d7e03b92 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:47.872321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1049/2122] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:47 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:47.880500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bc493-fbf2-410a-b64d-3d5356f7a1db admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:47.890283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0cdb3-184b-4441-b732-9091d7e03b92 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:47.892010 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-74f0cdb3-184b-4441-b732-9091d7e03b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.892290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bc493-fbf2-410a-b64d-3d5356f7a1db admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:47.979214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0cdb3-184b-4441-b732-9091d7e03b92 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:47.983558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:47.984626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.985134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.985453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.987308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:47.992857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab35ea-02b9-43e1-973b-717b324ef067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:47.998667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:47.998667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab35ea-02b9-43e1-973b-717b324ef067 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:47.998667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab35ea-02b9-43e1-973b-717b324ef067 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:47.998667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab35ea-02b9-43e1-973b-717b324ef067 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:47.998667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab35ea-02b9-43e1-973b-717b324ef067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:47.999193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab35ea-02b9-43e1-973b-717b324ef067 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:48.062842 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:48.064679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:48.091557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1050/2123] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:48.095929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1074/2124] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:47 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:48.096071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:48.096830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:48.097218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab35ea-02b9-43e1-973b-717b324ef067 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:48.113568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab35ea-02b9-43e1-973b-717b324ef067 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:48.126677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80087fe6-e928-43e8-b47f-61dfcd385ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:48.129403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85033049-836a-4777-9d65-b7aa3ba6c259 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:48.129683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80087fe6-e928-43e8-b47f-61dfcd385ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:48.130006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80087fe6-e928-43e8-b47f-61dfcd385ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:48.130612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80087fe6-e928-43e8-b47f-61dfcd385ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:48.132486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80087fe6-e928-43e8-b47f-61dfcd385ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:48.134145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80087fe6-e928-43e8-b47f-61dfcd385ce6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:48.138398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:48.142450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:48.142450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:48.142450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] PATH_INFO: `/v3/projects/b291c2cbdcf8476a82a651414daa676f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:48.142450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b291c2cbdcf8476a82a651414daa676f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:48.143357 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:48.159314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1075/2125] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 09:59:48.161156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80087fe6-e928-43e8-b47f-61dfcd385ce6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:48.177508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80087fe6-e928-43e8-b47f-61dfcd385ce6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:48.183293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12060065-72e0-40ee-939f-78c46aea028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:48.184590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12060065-72e0-40ee-939f-78c46aea028f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:48.184885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12060065-72e0-40ee-939f-78c46aea028f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:48.185168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12060065-72e0-40ee-939f-78c46aea028f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:48.186131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12060065-72e0-40ee-939f-78c46aea028f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:48.188180 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12060065-72e0-40ee-939f-78c46aea028f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:48.206156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1076/2126] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:48.206957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12060065-72e0-40ee-939f-78c46aea028f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:48.218338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12060065-72e0-40ee-939f-78c46aea028f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:48.244199 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3739b0a-9a90-4802-8068-809ea53833d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:48.247139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3739b0a-9a90-4802-8068-809ea53833d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:48.247139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3739b0a-9a90-4802-8068-809ea53833d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:48.247139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3739b0a-9a90-4802-8068-809ea53833d7 admin admin] PATH_INFO: `/v3/projects/31b7ad0a70d34261b6b8aafcdbc2c5c7/users/b37fcf4ca8984d0cb3582410324c4901/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:48.266324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3739b0a-9a90-4802-8068-809ea53833d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b7ad0a70d34261b6b8aafcdbc2c5c7, user_id=b37fcf4ca8984d0cb3582410324c4901, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:48.269150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3739b0a-9a90-4802-8068-809ea53833d7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:48.445749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1077/2127] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:48 2022] PUT /identity/v3/projects/31b7ad0a70d34261b6b8aafcdbc2c5c7/users/b37fcf4ca8984d0cb3582410324c4901/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:48.446476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b291c2cbdcf8476a82a651414daa676f', 'request_id': 'req-2fce30b8-af84-49dc-bff2-090f6e06dc95'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:48.448605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775cb3d-2d8e-4094-a858-79b1bbe1ac38 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:48.448904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775cb3d-2d8e-4094-a858-79b1bbe1ac38 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:48.449188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775cb3d-2d8e-4094-a858-79b1bbe1ac38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:48.452762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b291c2cbdcf8476a82a651414daa676f', 'request_id': 'req-2fce30b8-af84-49dc-bff2-090f6e06dc95'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:48.459316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:48.477700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7775cb3d-2d8e-4094-a858-79b1bbe1ac38 None None] MFA Rules not processed for user `b37fcf4ca8984d0cb3582410324c4901`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:48.478944 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7775cb3d-2d8e-4094-a858-79b1bbe1ac38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:48.544855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b291c2cbdcf8476a82a651414daa676f', 'request_id': 'req-2fce30b8-af84-49dc-bff2-090f6e06dc95'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:48.552490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1078/2128] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:48.569444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] The token cache is being invalidate because project b291c2cbdcf8476a82a651414daa676f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:48.569710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:48.570565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:48.572382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1051/2129] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:48 2022] DELETE /identity/v3/projects/b291c2cbdcf8476a82a651414daa676f => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:48.576890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775cb3d-2d8e-4094-a858-79b1bbe1ac38 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:48.630546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:48.646622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775cb3d-2d8e-4094-a858-79b1bbe1ac38 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:48.653087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:48.654008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:48.654294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:48.654569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] PATH_INFO: `/v3/users/bca5fde7191c4f76b8489caf62483d3d` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:48.655799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca5fde7191c4f76b8489caf62483d3d)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:48.657798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:48.811462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:48.813071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:48.869534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce30b8-af84-49dc-bff2-090f6e06dc95 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:48.873099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:48.873741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:48.874109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:48.874398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:48.876214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:48.887550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:48.889061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:48.897762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1052/2130] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:48.919242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca5fde7191c4f76b8489caf62483d3d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:48.919972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca5fde7191c4f76b8489caf62483d3d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:48.927433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca5fde7191c4f76b8489caf62483d3d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:48.940436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca5fde7191c4f76b8489caf62483d3d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:49.104166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-146bc173-4fc2-432f-8b42-5e0e71d95e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca5fde7191c4f76b8489caf62483d3d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:49.111264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1079/2131] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:48 2022] DELETE /identity/v3/users/bca5fde7191c4f76b8489caf62483d3d => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:49.648395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:49.649846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:49.673611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e338396-c325-40d2-8c1a-483f5c89bad8 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:49.746996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:49.747855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:49.748129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:49.748402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] PATH_INFO: `/v3/projects/fa08ecb75f6b48768741d9d27a0263fb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:49.749563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa08ecb75f6b48768741d9d27a0263fb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:49.751502 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:49.890599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa08ecb75f6b48768741d9d27a0263fb', 'request_id': 'req-cbd8df4a-a4da-4316-889a-565a62337578'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:49.896040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa08ecb75f6b48768741d9d27a0263fb', 'request_id': 'req-cbd8df4a-a4da-4316-889a-565a62337578'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:50.007371 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa08ecb75f6b48768741d9d27a0263fb', 'request_id': 'req-cbd8df4a-a4da-4316-889a-565a62337578'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:50.009784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:50.020474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] The token cache is being invalidate because project fa08ecb75f6b48768741d9d27a0263fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:50.020760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:50.021397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cbd8df4a-a4da-4316-889a-565a62337578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:50.023170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1053/2132] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:49 2022] DELETE /identity/v3/projects/fa08ecb75f6b48768741d9d27a0263fb => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:50.028590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1a804-a482-491d-b8e3-a4fa4369aaed None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.029032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1a804-a482-491d-b8e3-a4fa4369aaed None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.029269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1a804-a482-491d-b8e3-a4fa4369aaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.054317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-97b1a804-a482-491d-b8e3-a4fa4369aaed None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:50.054317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-97b1a804-a482-491d-b8e3-a4fa4369aaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:50.115839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1080/2133] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:50.123434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182913-d5d1-46d6-bf80-95483dcda240 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.123727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182913-d5d1-46d6-bf80-95483dcda240 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.124011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182913-d5d1-46d6-bf80-95483dcda240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.144246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-94182913-d5d1-46d6-bf80-95483dcda240 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:50.145392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-94182913-d5d1-46d6-bf80-95483dcda240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:50.153993 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1054/2134] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:50.158334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1a804-a482-491d-b8e3-a4fa4369aaed None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:50.168635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1a804-a482-491d-b8e3-a4fa4369aaed None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:50.173604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1287db-8c27-4cef-8b81-faf57a4be57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:50.174297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1287db-8c27-4cef-8b81-faf57a4be57c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.174583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1287db-8c27-4cef-8b81-faf57a4be57c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.174858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1287db-8c27-4cef-8b81-faf57a4be57c admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.175588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1287db-8c27-4cef-8b81-faf57a4be57c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:50.175896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1287db-8c27-4cef-8b81-faf57a4be57c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:50.177501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1287db-8c27-4cef-8b81-faf57a4be57c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:50.189536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1081/2135] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:50.194168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94182913-d5d1-46d6-bf80-95483dcda240 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:50.203206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94182913-d5d1-46d6-bf80-95483dcda240 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:50.207841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374726d-58b6-469d-88fb-a5c66450c0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:50.208482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374726d-58b6-469d-88fb-a5c66450c0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.208763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374726d-58b6-469d-88fb-a5c66450c0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.209036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374726d-58b6-469d-88fb-a5c66450c0b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.209727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374726d-58b6-469d-88fb-a5c66450c0b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:50.211697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374726d-58b6-469d-88fb-a5c66450c0b2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:50.345762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1055/2136] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:50 2022] POST /identity/v3/projects => generated 399 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:50.350067 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1287db-8c27-4cef-8b81-faf57a4be57c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:50.360870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1287db-8c27-4cef-8b81-faf57a4be57c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:50.415792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b98e98-56d4-4f70-9ced-d896b621b23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:50.416379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b98e98-56d4-4f70-9ced-d896b621b23c admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.416658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b98e98-56d4-4f70-9ced-d896b621b23c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.416934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b98e98-56d4-4f70-9ced-d896b621b23c admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.418164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b98e98-56d4-4f70-9ced-d896b621b23c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:50.420089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b98e98-56d4-4f70-9ced-d896b621b23c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:50.527443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd943265-3d5e-41c0-baa5-6585cc8cf94a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.527741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd943265-3d5e-41c0-baa5-6585cc8cf94a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.528024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd943265-3d5e-41c0-baa5-6585cc8cf94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.546028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bd943265-3d5e-41c0-baa5-6585cc8cf94a None None] MFA Rules not processed for user `3dd300436c9545faa56d60ac290bc987`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:50.547224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bd943265-3d5e-41c0-baa5-6585cc8cf94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:50.587570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1082/2137] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:50 2022] POST /identity/v3/users => generated 300 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:50.592900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b98e98-56d4-4f70-9ced-d896b621b23c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:50.602930 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b98e98-56d4-4f70-9ced-d896b621b23c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:50.612304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1056/2138] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:50.625590 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7d1f5-1cfb-47ce-a636-3a902ca4b051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:50.626381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7d1f5-1cfb-47ce-a636-3a902ca4b051 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.626736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7d1f5-1cfb-47ce-a636-3a902ca4b051 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.627086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7d1f5-1cfb-47ce-a636-3a902ca4b051 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.627967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7d1f5-1cfb-47ce-a636-3a902ca4b051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:50.629736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7d1f5-1cfb-47ce-a636-3a902ca4b051 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:50.641276 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1083/2139] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:50.644742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd943265-3d5e-41c0-baa5-6585cc8cf94a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:50.656277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd943265-3d5e-41c0-baa5-6585cc8cf94a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:50.662271 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4608f4-f063-4e7d-80d0-669a8363224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:50.663980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4608f4-f063-4e7d-80d0-669a8363224f admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.664282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4608f4-f063-4e7d-80d0-669a8363224f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.664579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4608f4-f063-4e7d-80d0-669a8363224f admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.665334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4608f4-f063-4e7d-80d0-669a8363224f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:50.667742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4608f4-f063-4e7d-80d0-669a8363224f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:50.679138 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1057/2140] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:50.683486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7d1f5-1cfb-47ce-a636-3a902ca4b051 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:50.693657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7d1f5-1cfb-47ce-a636-3a902ca4b051 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:50.698681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17b37-269a-401c-8764-2272f41eb917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:50.699823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17b37-269a-401c-8764-2272f41eb917 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.700123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17b37-269a-401c-8764-2272f41eb917 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.700412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17b37-269a-401c-8764-2272f41eb917 admin admin] PATH_INFO: `/v3/projects/a25631f160d94c568be96be9173546d6/users/9778e938b89843be8dc68d41d916aba5/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.732071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d17b37-269a-401c-8764-2272f41eb917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25631f160d94c568be96be9173546d6, user_id=9778e938b89843be8dc68d41d916aba5, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:50.735032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d17b37-269a-401c-8764-2272f41eb917 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:50.762587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:50.831671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1084/2141] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:50 2022] PUT /identity/v3/projects/a25631f160d94c568be96be9173546d6/users/9778e938b89843be8dc68d41d916aba5/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:50.844753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07250a-b52d-42a2-b1e1-023bfde1a2cf None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:50.849978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07250a-b52d-42a2-b1e1-023bfde1a2cf None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:50.849978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07250a-b52d-42a2-b1e1-023bfde1a2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:50.886280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5f07250a-b52d-42a2-b1e1-023bfde1a2cf None None] MFA Rules not processed for user `9778e938b89843be8dc68d41d916aba5`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:50.887803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5f07250a-b52d-42a2-b1e1-023bfde1a2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:50.956534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1058/2142] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:50.959482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17b37-269a-401c-8764-2272f41eb917 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:51.029608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17b37-269a-401c-8764-2272f41eb917 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:51.034542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d86a4-c117-4498-b112-beab74eaf2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:51.039268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d86a4-c117-4498-b112-beab74eaf2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:51.039268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d86a4-c117-4498-b112-beab74eaf2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:51.039268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d86a4-c117-4498-b112-beab74eaf2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:51.039268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d86a4-c117-4498-b112-beab74eaf2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:51.039268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d86a4-c117-4498-b112-beab74eaf2b9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:51.193175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1085/2143] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:50 2022] POST /identity/v3/projects => generated 401 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:51.197112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07250a-b52d-42a2-b1e1-023bfde1a2cf None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:51.232300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07250a-b52d-42a2-b1e1-023bfde1a2cf None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:51.288035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25190b6-121b-4be6-bde3-06c557a9e8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:51.288804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25190b6-121b-4be6-bde3-06c557a9e8ab admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:51.289192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25190b6-121b-4be6-bde3-06c557a9e8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:51.289542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25190b6-121b-4be6-bde3-06c557a9e8ab admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:51.290975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25190b6-121b-4be6-bde3-06c557a9e8ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:51.293100 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25190b6-121b-4be6-bde3-06c557a9e8ab admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:51.391970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1059/2144] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:51 2022] POST /identity/v3/users => generated 307 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 09:59:51.396036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d86a4-c117-4498-b112-beab74eaf2b9 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:51.406418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d86a4-c117-4498-b112-beab74eaf2b9 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:51.411375 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c88ad-9a3e-4a7b-8b38-e1f090615f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:51.419754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c88ad-9a3e-4a7b-8b38-e1f090615f77 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:51.419754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c88ad-9a3e-4a7b-8b38-e1f090615f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:51.419754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c88ad-9a3e-4a7b-8b38-e1f090615f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:51.419754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c88ad-9a3e-4a7b-8b38-e1f090615f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:51.419754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c88ad-9a3e-4a7b-8b38-e1f090615f77 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:51.426451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1086/2145] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:51.430756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25190b6-121b-4be6-bde3-06c557a9e8ab admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:51.442664 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25190b6-121b-4be6-bde3-06c557a9e8ab admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:51.447728 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d181d-b5dc-4c2d-9916-b83ea80ae770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:51.449289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d181d-b5dc-4c2d-9916-b83ea80ae770 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:51.449628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d181d-b5dc-4c2d-9916-b83ea80ae770 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:51.450209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d181d-b5dc-4c2d-9916-b83ea80ae770 admin admin] PATH_INFO: `/v3/projects/bff41148751242268963e5d000624fc7/users/29d72288712a4206bf16a8afb0189cf7/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:51.466803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d181d-b5dc-4c2d-9916-b83ea80ae770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff41148751242268963e5d000624fc7, user_id=29d72288712a4206bf16a8afb0189cf7, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:51.470289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d181d-b5dc-4c2d-9916-b83ea80ae770 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:51.626827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1060/2146] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:51 2022] PUT /identity/v3/projects/bff41148751242268963e5d000624fc7/users/29d72288712a4206bf16a8afb0189cf7/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:51.631189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c88ad-9a3e-4a7b-8b38-e1f090615f77 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:51.642459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c88ad-9a3e-4a7b-8b38-e1f090615f77 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:51.704542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6d5ac-969e-4788-96df-d196fe3068bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:51.705301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6d5ac-969e-4788-96df-d196fe3068bd admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:51.705667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6d5ac-969e-4788-96df-d196fe3068bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:51.706042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6d5ac-969e-4788-96df-d196fe3068bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:51.707017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6d5ac-969e-4788-96df-d196fe3068bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:51.709048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6d5ac-969e-4788-96df-d196fe3068bd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:51.720904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1087/2147] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:51.725467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d181d-b5dc-4c2d-9916-b83ea80ae770 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:51.727458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:51.739376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d181d-b5dc-4c2d-9916-b83ea80ae770 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:51.744183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b02fb7-7a15-440e-81c4-ca07f1032ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:51.745073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b02fb7-7a15-440e-81c4-ca07f1032ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:51.745937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b02fb7-7a15-440e-81c4-ca07f1032ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:51.746259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b02fb7-7a15-440e-81c4-ca07f1032ff8 admin admin] PATH_INFO: `/v3/projects/bff41148751242268963e5d000624fc7/users/29d72288712a4206bf16a8afb0189cf7/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:51.748187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b02fb7-7a15-440e-81c4-ca07f1032ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff41148751242268963e5d000624fc7, user_id=29d72288712a4206bf16a8afb0189cf7, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:51.750878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b02fb7-7a15-440e-81c4-ca07f1032ff8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:51.763152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1061/2148] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:51 2022] PUT /identity/v3/projects/bff41148751242268963e5d000624fc7/users/29d72288712a4206bf16a8afb0189cf7/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:51.774363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b3f67-5b34-436c-83cc-08e62db69370 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:51.774657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b3f67-5b34-436c-83cc-08e62db69370 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:51.774938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b3f67-5b34-436c-83cc-08e62db69370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:51.803094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b95b3f67-5b34-436c-83cc-08e62db69370 None None] MFA Rules not processed for user `29d72288712a4206bf16a8afb0189cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:51.804323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b95b3f67-5b34-436c-83cc-08e62db69370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:51.901127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1088/2149] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:51.935808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b02fb7-7a15-440e-81c4-ca07f1032ff8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:51.939131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-90b02fb7-7a15-440e-81c4-ca07f1032ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:52.001269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b02fb7-7a15-440e-81c4-ca07f1032ff8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:52.005135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05226e1c-c654-4118-aa72-b8dcfcd74080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:52.005938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05226e1c-c654-4118-aa72-b8dcfcd74080 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:52.006293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05226e1c-c654-4118-aa72-b8dcfcd74080 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:52.006661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05226e1c-c654-4118-aa72-b8dcfcd74080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:52.016155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05226e1c-c654-4118-aa72-b8dcfcd74080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:52.017703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05226e1c-c654-4118-aa72-b8dcfcd74080 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:52.028793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1062/2150] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:52.340477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b3f67-5b34-436c-83cc-08e62db69370 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:52.349999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b3f67-5b34-436c-83cc-08e62db69370 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:52.356472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c97cd-df4f-4934-8439-d055417e9bf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:52.357132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c97cd-df4f-4934-8439-d055417e9bf3 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:52.357419 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c97cd-df4f-4934-8439-d055417e9bf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:52.357751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c97cd-df4f-4934-8439-d055417e9bf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:52.416401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c97cd-df4f-4934-8439-d055417e9bf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:52.418461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c97cd-df4f-4934-8439-d055417e9bf3 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:52.427450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1089/2151] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:52.531618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:53.616948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:54.550614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:55.154629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:55.564478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05226e1c-c654-4118-aa72-b8dcfcd74080 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:55.566086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-05226e1c-c654-4118-aa72-b8dcfcd74080 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:55.635369 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05226e1c-c654-4118-aa72-b8dcfcd74080 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:55.640899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:55.641774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:55.642091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:55.642371 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] PATH_INFO: `/v3/users/6122c5b2981a4018a8c93dfa72b65919` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:55.643568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6122c5b2981a4018a8c93dfa72b65919)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:55.645510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:55.870059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6122c5b2981a4018a8c93dfa72b65919'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:55.870731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6122c5b2981a4018a8c93dfa72b65919'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:55.882562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6122c5b2981a4018a8c93dfa72b65919'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:55.889931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6122c5b2981a4018a8c93dfa72b65919'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:55.976645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6122c5b2981a4018a8c93dfa72b65919'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:55.984281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1063/2152] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:55 2022] DELETE /identity/v3/users/6122c5b2981a4018a8c93dfa72b65919 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:55.988364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:55.988938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82c80e-d0b7-423a-afd2-e5e647e81e45 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:55.989233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82c80e-d0b7-423a-afd2-e5e647e81e45 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:55.989510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82c80e-d0b7-423a-afd2-e5e647e81e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:56.008584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-aa82c80e-d0b7-423a-afd2-e5e647e81e45 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:56.009778 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aa82c80e-d0b7-423a-afd2-e5e647e81e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:56.071650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1090/2153] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:56.082008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:56.083462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:56.146888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d1d40-ef6f-4d4a-a7d2-72d2037bf290 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:56.150538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab953e-b0af-47f5-9d26-dcd2b29b2526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:56.155835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab953e-b0af-47f5-9d26-dcd2b29b2526 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:56.155835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab953e-b0af-47f5-9d26-dcd2b29b2526 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:56.155835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab953e-b0af-47f5-9d26-dcd2b29b2526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:56.162397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab953e-b0af-47f5-9d26-dcd2b29b2526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:56.163890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab953e-b0af-47f5-9d26-dcd2b29b2526 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:56.183753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1064/2154] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:56.701004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82c80e-d0b7-423a-afd2-e5e647e81e45 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:56.703305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aa82c80e-d0b7-423a-afd2-e5e647e81e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:56.741453 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82c80e-d0b7-423a-afd2-e5e647e81e45 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:56.746949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:56.748076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:56.748696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:56.749042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] PATH_INFO: `/v3/projects/5e7e805c429e4b61a9530da191f327a2` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:56.753326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e7e805c429e4b61a9530da191f327a2)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:56.754931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:56.877709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e7e805c429e4b61a9530da191f327a2', 'request_id': 'req-e8982496-0a8c-4566-a4c5-4194bed667ef'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:56.883567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e7e805c429e4b61a9530da191f327a2', 'request_id': 'req-e8982496-0a8c-4566-a4c5-4194bed667ef'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:56.974006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:57.024352 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e7e805c429e4b61a9530da191f327a2', 'request_id': 'req-e8982496-0a8c-4566-a4c5-4194bed667ef'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:57.047382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] The token cache is being invalidate because project 5e7e805c429e4b61a9530da191f327a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:57.047672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:57.048330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:57.049898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1091/2155] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:56 2022] DELETE /identity/v3/projects/5e7e805c429e4b61a9530da191f327a2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:57.053983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab953e-b0af-47f5-9d26-dcd2b29b2526 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:57.063675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab953e-b0af-47f5-9d26-dcd2b29b2526 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:57.121573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:57.121971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:57.122408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:57.122870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] PATH_INFO: `/v3/users/d9bbb5bf7b234109a00256d5ba6866cb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:57.124624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bbb5bf7b234109a00256d5ba6866cb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:57.126736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:57.387230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bbb5bf7b234109a00256d5ba6866cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:57.387903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bbb5bf7b234109a00256d5ba6866cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:57.399983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bbb5bf7b234109a00256d5ba6866cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:57.406905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bbb5bf7b234109a00256d5ba6866cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:57.492668 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bbb5bf7b234109a00256d5ba6866cb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:57.499535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1065/2156] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:57 2022] DELETE /identity/v3/users/d9bbb5bf7b234109a00256d5ba6866cb => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:57.570940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:58.025849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:58.028003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:58.044646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8982496-0a8c-4566-a4c5-4194bed667ef admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:58.107505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:58.109503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:58.124311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:58.125245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:58.125560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:58.126260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] PATH_INFO: `/v3/projects/91444d95f649480ba256a4dd5799aa72` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:58.129594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] RBAC: Authorizing `identity:delete_project(project_id=91444d95f649480ba256a4dd5799aa72)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:58.131635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:58.181750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fa0e-1bd0-4ef9-9124-fe4e4181f46d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:58.187587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:58.188826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:58.188826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:58.189076 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] PATH_INFO: `/v3/users/9778e938b89843be8dc68d41d916aba5` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:58.190441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9778e938b89843be8dc68d41d916aba5)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:58.192415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:58.428206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9778e938b89843be8dc68d41d916aba5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.428890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9778e938b89843be8dc68d41d916aba5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.429911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91444d95f649480ba256a4dd5799aa72', 'request_id': 'req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.439568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91444d95f649480ba256a4dd5799aa72', 'request_id': 'req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.444265 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9778e938b89843be8dc68d41d916aba5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.451279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9778e938b89843be8dc68d41d916aba5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.547731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91444d95f649480ba256a4dd5799aa72', 'request_id': 'req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.560808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] The token cache is being invalidate because project 91444d95f649480ba256a4dd5799aa72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:58.561104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.561967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f258418d-84f2-4d9b-a2d4-ebb3ac47154a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.569666 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1092/2157] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:58 2022] DELETE /identity/v3/projects/91444d95f649480ba256a4dd5799aa72 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:58.575580 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b017a3-47b8-44ff-93cd-222d4efe6de6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:58.575956 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b017a3-47b8-44ff-93cd-222d4efe6de6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:58.576350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b017a3-47b8-44ff-93cd-222d4efe6de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:58.601121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-32b017a3-47b8-44ff-93cd-222d4efe6de6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:58.602457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-32b017a3-47b8-44ff-93cd-222d4efe6de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:58.617144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4efec07a-036d-4f1b-8afd-44593c1e8847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9778e938b89843be8dc68d41d916aba5'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:58.628991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1066/2158] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:58 2022] DELETE /identity/v3/users/9778e938b89843be8dc68d41d916aba5 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:58.630990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb98de9-5145-45d6-a337-753c4521ffc2 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:58.631385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb98de9-5145-45d6-a337-753c4521ffc2 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:58.631671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb98de9-5145-45d6-a337-753c4521ffc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:58.650628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-dfb98de9-5145-45d6-a337-753c4521ffc2 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:58.652666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dfb98de9-5145-45d6-a337-753c4521ffc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:58.657969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 09:59:58.681945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1093/2159] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:58.683959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12766a-4bb2-4ff3-831f-5e52ddf2832c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:58.684251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12766a-4bb2-4ff3-831f-5e52ddf2832c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:58.684581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12766a-4bb2-4ff3-831f-5e52ddf2832c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:58.708697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fd12766a-4bb2-4ff3-831f-5e52ddf2832c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:58.711850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fd12766a-4bb2-4ff3-831f-5e52ddf2832c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:58.738727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1067/2160] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:58.749460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb98de9-5145-45d6-a337-753c4521ffc2 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:58.750911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dfb98de9-5145-45d6-a337-753c4521ffc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:58.779266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1094/2161] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 09:59:58.783496 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12766a-4bb2-4ff3-831f-5e52ddf2832c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:58.792464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12766a-4bb2-4ff3-831f-5e52ddf2832c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:58.797105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832b52b-85c9-4d16-a882-3f9d7c6abb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:58.797748 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832b52b-85c9-4d16-a882-3f9d7c6abb56 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:58.798065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832b52b-85c9-4d16-a882-3f9d7c6abb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:58.798344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832b52b-85c9-4d16-a882-3f9d7c6abb56 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:58.799096 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832b52b-85c9-4d16-a882-3f9d7c6abb56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 09:59:58.799403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832b52b-85c9-4d16-a882-3f9d7c6abb56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:58.801020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832b52b-85c9-4d16-a882-3f9d7c6abb56 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:58.810903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb98de9-5145-45d6-a337-753c4521ffc2 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:58.814962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1095/2162] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 09:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 09:59:58.817562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e9e56-7064-48e5-b038-0cd76b2cb1c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:58.817776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832b52b-85c9-4d16-a882-3f9d7c6abb56 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:58.818237 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e9e56-7064-48e5-b038-0cd76b2cb1c8 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:58.818521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e9e56-7064-48e5-b038-0cd76b2cb1c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:58.818801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e9e56-7064-48e5-b038-0cd76b2cb1c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:58.828078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832b52b-85c9-4d16-a882-3f9d7c6abb56 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:58.828447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e9e56-7064-48e5-b038-0cd76b2cb1c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:58.829923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e9e56-7064-48e5-b038-0cd76b2cb1c8 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:58.832784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555af8f-408b-4918-a486-f86880eda310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:58.833431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555af8f-408b-4918-a486-f86880eda310 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:58.833713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555af8f-408b-4918-a486-f86880eda310 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:58.834050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555af8f-408b-4918-a486-f86880eda310 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:58.835304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555af8f-408b-4918-a486-f86880eda310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:58.836782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555af8f-408b-4918-a486-f86880eda310 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:58.840651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1068/2163] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 09:59:59.028503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1096/2164] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 09:59:58 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:59.029962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e9e56-7064-48e5-b038-0cd76b2cb1c8 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:59.040687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e9e56-7064-48e5-b038-0cd76b2cb1c8 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:59.100877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc1579-56cb-4f52-b662-136c794bb241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:59.101478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc1579-56cb-4f52-b662-136c794bb241 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:59.101760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc1579-56cb-4f52-b662-136c794bb241 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:59.102070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc1579-56cb-4f52-b662-136c794bb241 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:59.102801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc1579-56cb-4f52-b662-136c794bb241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:59.104938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc1579-56cb-4f52-b662-136c794bb241 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:59.213055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1069/2165] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 09:59:59 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 09:59:59.217233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555af8f-408b-4918-a486-f86880eda310 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:59.227835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555af8f-408b-4918-a486-f86880eda310 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:59.234986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c929b-1f33-4d57-9050-2132c3ec06c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:59.235696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c929b-1f33-4d57-9050-2132c3ec06c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:59.235979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c929b-1f33-4d57-9050-2132c3ec06c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:59.236255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c929b-1f33-4d57-9050-2132c3ec06c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:59.236986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c929b-1f33-4d57-9050-2132c3ec06c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:59.238679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c929b-1f33-4d57-9050-2132c3ec06c5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:59.252272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1097/2166] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:59.253741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc1579-56cb-4f52-b662-136c794bb241 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:59.264086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc1579-56cb-4f52-b662-136c794bb241 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:59.268954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede84670-8c28-4a64-8479-652b9307ce07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:59.269620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede84670-8c28-4a64-8479-652b9307ce07 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:59.269964 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede84670-8c28-4a64-8479-652b9307ce07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:59.270246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede84670-8c28-4a64-8479-652b9307ce07 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:59.270983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede84670-8c28-4a64-8479-652b9307ce07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:59.272640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede84670-8c28-4a64-8479-652b9307ce07 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:59.283547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1070/2167] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 09:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 09:59:59.287670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c929b-1f33-4d57-9050-2132c3ec06c5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:59.298126 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c929b-1f33-4d57-9050-2132c3ec06c5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:59.302620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2c9b0-48e1-49b0-af48-a7cf1cc4c3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:59.303494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2c9b0-48e1-49b0-af48-a7cf1cc4c3e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:59.303777 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2c9b0-48e1-49b0-af48-a7cf1cc4c3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:59.304050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2c9b0-48e1-49b0-af48-a7cf1cc4c3e9 admin admin] PATH_INFO: `/v3/projects/6d91dbf6be3e4ce3b6d34cdc6175ecf0/users/07304f6f3d354b36aa150f8dc5917017/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:59.320898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2c9b0-48e1-49b0-af48-a7cf1cc4c3e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d91dbf6be3e4ce3b6d34cdc6175ecf0, user_id=07304f6f3d354b36aa150f8dc5917017, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:59.323637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2c9b0-48e1-49b0-af48-a7cf1cc4c3e9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:59.380417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede84670-8c28-4a64-8479-652b9307ce07 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:59.384121 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ede84670-8c28-4a64-8479-652b9307ce07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:59.395967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede84670-8c28-4a64-8479-652b9307ce07 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:59.400982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:59.401953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:59.402241 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:59.402539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] PATH_INFO: `/v3/projects/a25631f160d94c568be96be9173546d6` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:59.404469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25631f160d94c568be96be9173546d6)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:59.406908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:59.419144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1098/2168] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 09:59:59 2022] PUT /identity/v3/projects/6d91dbf6be3e4ce3b6d34cdc6175ecf0/users/07304f6f3d354b36aa150f8dc5917017/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 09:59:59.429391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48931ddf-00ae-4779-9917-167595cc433c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:59.429679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48931ddf-00ae-4779-9917-167595cc433c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:59.429992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48931ddf-00ae-4779-9917-167595cc433c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:59.456512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-48931ddf-00ae-4779-9917-167595cc433c None None] MFA Rules not processed for user `07304f6f3d354b36aa150f8dc5917017`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 09:59:59.457716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-48931ddf-00ae-4779-9917-167595cc433c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:59.522814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1099/2169] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 09:59:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 09:59:59.536083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48931ddf-00ae-4779-9917-167595cc433c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:59.539788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-48931ddf-00ae-4779-9917-167595cc433c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:59.595128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48931ddf-00ae-4779-9917-167595cc433c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:59.598699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90492e4-45bb-4432-af72-bfa5a7f63337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:59.599346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90492e4-45bb-4432-af72-bfa5a7f63337 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:59.599623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90492e4-45bb-4432-af72-bfa5a7f63337 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:59.599905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90492e4-45bb-4432-af72-bfa5a7f63337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:59.608988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90492e4-45bb-4432-af72-bfa5a7f63337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:59.610468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90492e4-45bb-4432-af72-bfa5a7f63337 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:59.619682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1100/2170] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 09:59:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 09:59:59.642702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90492e4-45bb-4432-af72-bfa5a7f63337 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:59.659195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25631f160d94c568be96be9173546d6', 'request_id': 'req-ef6f7155-0a49-421b-a8b1-2a80bd497a29'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:59.666971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25631f160d94c568be96be9173546d6', 'request_id': 'req-ef6f7155-0a49-421b-a8b1-2a80bd497a29'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:59.726709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90492e4-45bb-4432-af72-bfa5a7f63337 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:59.781877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25631f160d94c568be96be9173546d6', 'request_id': 'req-ef6f7155-0a49-421b-a8b1-2a80bd497a29'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:59.782260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:59.783120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:59.783398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:59.783669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] PATH_INFO: `/v3/users/07304f6f3d354b36aa150f8dc5917017` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:59.784864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07304f6f3d354b36aa150f8dc5917017)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:59.786805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:59.794791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] The token cache is being invalidate because project a25631f160d94c568be96be9173546d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 09:59:59.795077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:59.796730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 09:59:59.797535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1071/2171] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 09:59:59 2022] DELETE /identity/v3/projects/a25631f160d94c568be96be9173546d6 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 09:59:59.801352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 09:59:59.802702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 09:59:59.814880 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f7155-0a49-421b-a8b1-2a80bd497a29 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 09:59:59.819311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 09:59:59.820153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 09:59:59.820432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 09:59:59.820705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] PATH_INFO: `/v3/users/29d72288712a4206bf16a8afb0189cf7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 09:59:59.821848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d72288712a4206bf16a8afb0189cf7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 09:59:59.823822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 09:59:59.830132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:00.046457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07304f6f3d354b36aa150f8dc5917017'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.047070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07304f6f3d354b36aa150f8dc5917017'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.053837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07304f6f3d354b36aa150f8dc5917017'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.065268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07304f6f3d354b36aa150f8dc5917017'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.087822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d72288712a4206bf16a8afb0189cf7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.088476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d72288712a4206bf16a8afb0189cf7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.100864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d72288712a4206bf16a8afb0189cf7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.107578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d72288712a4206bf16a8afb0189cf7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.137267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5d54ef66-d80b-46b3-b89c-4b3e628a2a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07304f6f3d354b36aa150f8dc5917017'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.146765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1101/2172] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:59 2022] DELETE /identity/v3/users/07304f6f3d354b36aa150f8dc5917017 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:00.148452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0067a-db7d-479c-892c-7985227fb7eb None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:00.148741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0067a-db7d-479c-892c-7985227fb7eb None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:00.149046 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0067a-db7d-479c-892c-7985227fb7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:00.167798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4ce0067a-db7d-479c-892c-7985227fb7eb None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:00.169010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4ce0067a-db7d-479c-892c-7985227fb7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:00.218360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d72288712a4206bf16a8afb0189cf7'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:00.225079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1072/2173] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 09:59:59 2022] DELETE /identity/v3/users/29d72288712a4206bf16a8afb0189cf7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:00:00.230585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1102/2174] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:00.240700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:00.242314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:00.303807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e3cbb-9b4f-43d5-88ac-ebccbd77f82d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:00.308139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b30624-018f-4833-a1e0-4b59f346771e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:00.308793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b30624-018f-4833-a1e0-4b59f346771e service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:00.309080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b30624-018f-4833-a1e0-4b59f346771e service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:00.309351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b30624-018f-4833-a1e0-4b59f346771e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:00.389442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b30624-018f-4833-a1e0-4b59f346771e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:00.390955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b30624-018f-4833-a1e0-4b59f346771e service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:00.401994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1073/2175] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:00.838209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0067a-db7d-479c-892c-7985227fb7eb None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:00.842308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4ce0067a-db7d-479c-892c-7985227fb7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:00.872050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b30624-018f-4833-a1e0-4b59f346771e service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:00.873851 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a7b30624-018f-4833-a1e0-4b59f346771e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:00.882602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0067a-db7d-479c-892c-7985227fb7eb None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:00.891122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:00.892055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:00.892395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:00.892677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] PATH_INFO: `/v3/projects/bff41148751242268963e5d000624fc7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:00.894103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff41148751242268963e5d000624fc7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:00.894376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b30624-018f-4833-a1e0-4b59f346771e service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:00.896181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:00.907676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:00.909088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:00.909375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:00.909689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] PATH_INFO: `/v3/projects/6d91dbf6be3e4ce3b6d34cdc6175ecf0` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:00.915228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d91dbf6be3e4ce3b6d34cdc6175ecf0)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:00.917224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:00.973300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:01.052554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff41148751242268963e5d000624fc7', 'request_id': 'req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.058671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff41148751242268963e5d000624fc7', 'request_id': 'req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.085362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d91dbf6be3e4ce3b6d34cdc6175ecf0', 'request_id': 'req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.090726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d91dbf6be3e4ce3b6d34cdc6175ecf0', 'request_id': 'req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.167474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff41148751242268963e5d000624fc7', 'request_id': 'req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.167806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:01.179602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] The token cache is being invalidate because project bff41148751242268963e5d000624fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:01.179893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.180534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-145743c5-d387-41e8-b5e8-f3d8f33f6c4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.182165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1103/2176] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:00 2022] DELETE /identity/v3/projects/bff41148751242268963e5d000624fc7 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:01.187391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3c1c6-2f75-4fb5-b5ff-ac67480e6808 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.187703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3c1c6-2f75-4fb5-b5ff-ac67480e6808 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.187993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3c1c6-2f75-4fb5-b5ff-ac67480e6808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.205365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c4d3c1c6-2f75-4fb5-b5ff-ac67480e6808 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:01.206616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c4d3c1c6-2f75-4fb5-b5ff-ac67480e6808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:01.265319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1104/2177] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:01.267761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d91dbf6be3e4ce3b6d34cdc6175ecf0', 'request_id': 'req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.272741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf47a1-fa84-428f-9cca-3a4d30d19f63 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.273024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf47a1-fa84-428f-9cca-3a4d30d19f63 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.273302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf47a1-fa84-428f-9cca-3a4d30d19f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.279756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] The token cache is being invalidate because project 6d91dbf6be3e4ce3b6d34cdc6175ecf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:01.279953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.284398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d7b72315-7a5f-481d-9b7e-7ea7be2a1ab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:01.284398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1074/2178] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:00 2022] DELETE /identity/v3/projects/6d91dbf6be3e4ce3b6d34cdc6175ecf0 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:01.288038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d28c69-2acf-4514-96fb-1cf37fb4cace None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.288505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d28c69-2acf-4514-96fb-1cf37fb4cace None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.288857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d28c69-2acf-4514-96fb-1cf37fb4cace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.290999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9baf47a1-fa84-428f-9cca-3a4d30d19f63 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:01.292204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9baf47a1-fa84-428f-9cca-3a4d30d19f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:01.300288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1105/2179] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:01.304279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf47a1-fa84-428f-9cca-3a4d30d19f63 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.310370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-07d28c69-2acf-4514-96fb-1cf37fb4cace None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:01.311583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-07d28c69-2acf-4514-96fb-1cf37fb4cace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:01.314205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf47a1-fa84-428f-9cca-3a4d30d19f63 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.324030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1075/2180] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:01.326176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa589fd-ff9a-4232-b757-45985b864b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.326848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa589fd-ff9a-4232-b757-45985b864b56 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.327138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa589fd-ff9a-4232-b757-45985b864b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.327423 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa589fd-ff9a-4232-b757-45985b864b56 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.328158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa589fd-ff9a-4232-b757-45985b864b56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:00:01.328470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa589fd-ff9a-4232-b757-45985b864b56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.330159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa589fd-ff9a-4232-b757-45985b864b56 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.332886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf04ac-1c2d-4165-a571-1e104d037f4f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.333180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf04ac-1c2d-4165-a571-1e104d037f4f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.333460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf04ac-1c2d-4165-a571-1e104d037f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.350161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1106/2181] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:00:01.351301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2abf04ac-1c2d-4165-a571-1e104d037f4f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:01.352500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2abf04ac-1c2d-4165-a571-1e104d037f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:01.353100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa589fd-ff9a-4232-b757-45985b864b56 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.360427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1076/2182] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:01.362641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa589fd-ff9a-4232-b757-45985b864b56 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.364347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf04ac-1c2d-4165-a571-1e104d037f4f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.367194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fdc44-00e5-4c69-9867-bdce54647699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.367825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fdc44-00e5-4c69-9867-bdce54647699 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.368112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fdc44-00e5-4c69-9867-bdce54647699 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.368384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fdc44-00e5-4c69-9867-bdce54647699 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.369062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fdc44-00e5-4c69-9867-bdce54647699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.371608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fdc44-00e5-4c69-9867-bdce54647699 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.373610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf04ac-1c2d-4165-a571-1e104d037f4f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.378217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbecc3b-5a7f-47f8-970a-88ab8e3e447b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.378857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbecc3b-5a7f-47f8-970a-88ab8e3e447b admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.379139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbecc3b-5a7f-47f8-970a-88ab8e3e447b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.379419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbecc3b-5a7f-47f8-970a-88ab8e3e447b admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.380212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbecc3b-5a7f-47f8-970a-88ab8e3e447b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:00:01.380518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbecc3b-5a7f-47f8-970a-88ab8e3e447b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.382127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbecc3b-5a7f-47f8-970a-88ab8e3e447b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.394719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1077/2183] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:00:01.412862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbecc3b-5a7f-47f8-970a-88ab8e3e447b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.423251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbecc3b-5a7f-47f8-970a-88ab8e3e447b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.427911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00187d-aa66-4816-825b-be56c17952e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.428583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00187d-aa66-4816-825b-be56c17952e2 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.428865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00187d-aa66-4816-825b-be56c17952e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.429141 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00187d-aa66-4816-825b-be56c17952e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.429861 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00187d-aa66-4816-825b-be56c17952e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.431784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00187d-aa66-4816-825b-be56c17952e2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.520982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1107/2184] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:01.525143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fdc44-00e5-4c69-9867-bdce54647699 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.552485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1078/2185] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/projects => generated 369 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:01.562203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00187d-aa66-4816-825b-be56c17952e2 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.588033 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fdc44-00e5-4c69-9867-bdce54647699 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.634306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00187d-aa66-4816-825b-be56c17952e2 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.639626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166adee-1a74-4957-b5df-f3e2fda5428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.640270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166adee-1a74-4957-b5df-f3e2fda5428e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.640519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166adee-1a74-4957-b5df-f3e2fda5428e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.640787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166adee-1a74-4957-b5df-f3e2fda5428e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.641481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166adee-1a74-4957-b5df-f3e2fda5428e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.643716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166adee-1a74-4957-b5df-f3e2fda5428e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.657491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd4e95-8469-43e3-bba9-a6e453e87112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.662800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd4e95-8469-43e3-bba9-a6e453e87112 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.662800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd4e95-8469-43e3-bba9-a6e453e87112 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.662800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd4e95-8469-43e3-bba9-a6e453e87112 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.662800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd4e95-8469-43e3-bba9-a6e453e87112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.662800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd4e95-8469-43e3-bba9-a6e453e87112 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.681698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:01.734542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1079/2186] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:01.741315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166adee-1a74-4957-b5df-f3e2fda5428e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.753347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166adee-1a74-4957-b5df-f3e2fda5428e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.757959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee0bcb-17df-4ac7-bf9a-10c41495f245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.758612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0bcb-17df-4ac7-bf9a-10c41495f245 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.758899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0bcb-17df-4ac7-bf9a-10c41495f245 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.759184 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0bcb-17df-4ac7-bf9a-10c41495f245 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.759931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee0bcb-17df-4ac7-bf9a-10c41495f245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.761583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee0bcb-17df-4ac7-bf9a-10c41495f245 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.772754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1080/2187] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:01.777064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee0bcb-17df-4ac7-bf9a-10c41495f245 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.778352 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1108/2188] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/users => generated 291 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:01.782150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd4e95-8469-43e3-bba9-a6e453e87112 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.787388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee0bcb-17df-4ac7-bf9a-10c41495f245 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.793192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e40e1-af12-4cf9-b42c-9d651e6cc634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.793851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd4e95-8469-43e3-bba9-a6e453e87112 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.794003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e40e1-af12-4cf9-b42c-9d651e6cc634 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.794163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e40e1-af12-4cf9-b42c-9d651e6cc634 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.794436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e40e1-af12-4cf9-b42c-9d651e6cc634 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.795161 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e40e1-af12-4cf9-b42c-9d651e6cc634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.796909 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e40e1-af12-4cf9-b42c-9d651e6cc634 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.799016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953ac0b-1c2c-4dd9-a214-b74ca297a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.799466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953ac0b-1c2c-4dd9-a214-b74ca297a37c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.799751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953ac0b-1c2c-4dd9-a214-b74ca297a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.800031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953ac0b-1c2c-4dd9-a214-b74ca297a37c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.800761 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953ac0b-1c2c-4dd9-a214-b74ca297a37c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.802385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953ac0b-1c2c-4dd9-a214-b74ca297a37c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.808969 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1081/2189] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:01.813670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1109/2190] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:01.814461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e40e1-af12-4cf9-b42c-9d651e6cc634 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.817163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953ac0b-1c2c-4dd9-a214-b74ca297a37c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.824527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e40e1-af12-4cf9-b42c-9d651e6cc634 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.827804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953ac0b-1c2c-4dd9-a214-b74ca297a37c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.829421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1a331-8ad0-4db0-99e7-b42ed656a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.830327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1a331-8ad0-4db0-99e7-b42ed656a651 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.830615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1a331-8ad0-4db0-99e7-b42ed656a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.830891 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1a331-8ad0-4db0-99e7-b42ed656a651 admin admin] PATH_INFO: `/v3/projects/4718707db7fa48d3a4e711e0818fd861/users/c7c6f34b37b2424cad0aa958ed88b2b8/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.832754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea0a5f-ba34-47f8-9f24-b1c2c3f8fecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.833519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea0a5f-ba34-47f8-9f24-b1c2c3f8fecd admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.833829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea0a5f-ba34-47f8-9f24-b1c2c3f8fecd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.836816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea0a5f-ba34-47f8-9f24-b1c2c3f8fecd admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.836816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea0a5f-ba34-47f8-9f24-b1c2c3f8fecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.836816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea0a5f-ba34-47f8-9f24-b1c2c3f8fecd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.845773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1a331-8ad0-4db0-99e7-b42ed656a651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4718707db7fa48d3a4e711e0818fd861, user_id=c7c6f34b37b2424cad0aa958ed88b2b8, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.850680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1110/2191] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:01.850837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1a331-8ad0-4db0-99e7-b42ed656a651 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.851974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea0a5f-ba34-47f8-9f24-b1c2c3f8fecd admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:01.861739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea0a5f-ba34-47f8-9f24-b1c2c3f8fecd admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:01.870454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4816b-b02d-427f-86ef-5d52759e0f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:01.870454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4816b-b02d-427f-86ef-5d52759e0f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.870454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4816b-b02d-427f-86ef-5d52759e0f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.870454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4816b-b02d-427f-86ef-5d52759e0f5a admin admin] PATH_INFO: `/v3/projects/55e5440700564161b28cd3e21d7d64d3/users/c0e59127f52f415ea312d2b1a50b3762/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.882654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4816b-b02d-427f-86ef-5d52759e0f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e5440700564161b28cd3e21d7d64d3, user_id=c0e59127f52f415ea312d2b1a50b3762, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:01.885294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4816b-b02d-427f-86ef-5d52759e0f5a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:01.963626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1082/2192] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:01 2022] PUT /identity/v3/projects/4718707db7fa48d3a4e711e0818fd861/users/c7c6f34b37b2424cad0aa958ed88b2b8/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:01.965137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc6e04-cf45-4277-83ad-4c872cb33e02 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.965440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc6e04-cf45-4277-83ad-4c872cb33e02 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.965722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc6e04-cf45-4277-83ad-4c872cb33e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:01.983147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1111/2193] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:01 2022] PUT /identity/v3/projects/55e5440700564161b28cd3e21d7d64d3/users/c0e59127f52f415ea312d2b1a50b3762/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:00:01.994827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0f640-c1e4-433e-be2b-66bc095880b3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:01.995827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0f640-c1e4-433e-be2b-66bc095880b3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:01.996169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0f640-c1e4-433e-be2b-66bc095880b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:02.000788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f4fc6e04-cf45-4277-83ad-4c872cb33e02 None None] MFA Rules not processed for user `c7c6f34b37b2424cad0aa958ed88b2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:02.002062 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f4fc6e04-cf45-4277-83ad-4c872cb33e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:02.027630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2bd0f640-c1e4-433e-be2b-66bc095880b3 None None] MFA Rules not processed for user `c0e59127f52f415ea312d2b1a50b3762`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:02.028966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2bd0f640-c1e4-433e-be2b-66bc095880b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:02.083234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1083/2194] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:02.095054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc6e04-cf45-4277-83ad-4c872cb33e02 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:02.096428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f4fc6e04-cf45-4277-83ad-4c872cb33e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:02.099757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1112/2195] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:02.104105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0f640-c1e4-433e-be2b-66bc095880b3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:02.156491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc6e04-cf45-4277-83ad-4c872cb33e02 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:02.160099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d33ef-bb9e-4c09-820f-57af0d3c7c94 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd59a351971e94aa38a22e8113207e5b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:02.160690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d33ef-bb9e-4c09-820f-57af0d3c7c94 service glance] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:02.160982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d33ef-bb9e-4c09-820f-57af0d3c7c94 service glance] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:02.161261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d33ef-bb9e-4c09-820f-57af0d3c7c94 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:02.168882 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0f640-c1e4-433e-be2b-66bc095880b3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:02.170284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d33ef-bb9e-4c09-820f-57af0d3c7c94 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:02.171745 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d33ef-bb9e-4c09-820f-57af0d3c7c94 service glance] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:02.173622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd55f80-265a-4626-b3aa-7a7bc503ceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:02.174317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd55f80-265a-4626-b3aa-7a7bc503ceb5 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:02.174607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd55f80-265a-4626-b3aa-7a7bc503ceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:02.174889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd55f80-265a-4626-b3aa-7a7bc503ceb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:02.175587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd55f80-265a-4626-b3aa-7a7bc503ceb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:02.177577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd55f80-265a-4626-b3aa-7a7bc503ceb5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:02.180464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1084/2196] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:00:02.304864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1113/2197] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:02 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:02.305804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d33ef-bb9e-4c09-820f-57af0d3c7c94 service glance] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:02.317364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d33ef-bb9e-4c09-820f-57af0d3c7c94 service glance] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:02.378886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f18840-1a94-45c3-8e0a-915a73817bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:02.379597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f18840-1a94-45c3-8e0a-915a73817bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:02.380374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f18840-1a94-45c3-8e0a-915a73817bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:02.380726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f18840-1a94-45c3-8e0a-915a73817bd8 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:02.382340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f18840-1a94-45c3-8e0a-915a73817bd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:02.384295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f18840-1a94-45c3-8e0a-915a73817bd8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:02.499559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1085/2198] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:02 2022] POST /identity/v3/users => generated 298 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:02.503198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd55f80-265a-4626-b3aa-7a7bc503ceb5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:02.513827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd55f80-265a-4626-b3aa-7a7bc503ceb5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:02.518535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d056a-de8d-4604-90ed-49ad8524f03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:02.519228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d056a-de8d-4604-90ed-49ad8524f03a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:02.519562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d056a-de8d-4604-90ed-49ad8524f03a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:02.519842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d056a-de8d-4604-90ed-49ad8524f03a admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:02.520568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d056a-de8d-4604-90ed-49ad8524f03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:02.522191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d056a-de8d-4604-90ed-49ad8524f03a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:02.533426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1114/2199] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:02.537538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f18840-1a94-45c3-8e0a-915a73817bd8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:02.548811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f18840-1a94-45c3-8e0a-915a73817bd8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:02.553525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a611f-e95c-4ca7-ab61-bb27832165c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:02.554442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a611f-e95c-4ca7-ab61-bb27832165c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:02.554726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a611f-e95c-4ca7-ab61-bb27832165c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:02.555009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a611f-e95c-4ca7-ab61-bb27832165c9 admin admin] PATH_INFO: `/v3/projects/b294a85f88cc45e0968d55a0b15af1f4/users/c2d55d45bd3c4ac5acff1bd098fbf472/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:02.570218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a611f-e95c-4ca7-ab61-bb27832165c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b294a85f88cc45e0968d55a0b15af1f4, user_id=c2d55d45bd3c4ac5acff1bd098fbf472, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:02.574005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a611f-e95c-4ca7-ab61-bb27832165c9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:02.698744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1086/2200] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:02 2022] PUT /identity/v3/projects/b294a85f88cc45e0968d55a0b15af1f4/users/c2d55d45bd3c4ac5acff1bd098fbf472/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:00:02.702795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d056a-de8d-4604-90ed-49ad8524f03a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:02.715137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d056a-de8d-4604-90ed-49ad8524f03a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:02.778653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ab413-7649-4505-a342-4108ac4bff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:02.779322 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ab413-7649-4505-a342-4108ac4bff28 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:02.779602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ab413-7649-4505-a342-4108ac4bff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:02.779989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ab413-7649-4505-a342-4108ac4bff28 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:02.780718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ab413-7649-4505-a342-4108ac4bff28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:02.782340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ab413-7649-4505-a342-4108ac4bff28 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:02.792772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1115/2201] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:02.797116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a611f-e95c-4ca7-ab61-bb27832165c9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:02.807838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a611f-e95c-4ca7-ab61-bb27832165c9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:02.812540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595be814-6691-41fe-95f3-d653bef29680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:02.813450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be814-6691-41fe-95f3-d653bef29680 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:02.813726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be814-6691-41fe-95f3-d653bef29680 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:02.814041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be814-6691-41fe-95f3-d653bef29680 admin admin] PATH_INFO: `/v3/projects/b294a85f88cc45e0968d55a0b15af1f4/users/c2d55d45bd3c4ac5acff1bd098fbf472/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:02.815970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595be814-6691-41fe-95f3-d653bef29680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b294a85f88cc45e0968d55a0b15af1f4, user_id=c2d55d45bd3c4ac5acff1bd098fbf472, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:02.818913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595be814-6691-41fe-95f3-d653bef29680 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:02.831459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1087/2202] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:02 2022] PUT /identity/v3/projects/b294a85f88cc45e0968d55a0b15af1f4/users/c2d55d45bd3c4ac5acff1bd098fbf472/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:00:02.842106 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6875f-f62c-4d01-a29a-5d456a064779 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:02.842468 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6875f-f62c-4d01-a29a-5d456a064779 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:02.842820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6875f-f62c-4d01-a29a-5d456a064779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:02.861242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:02.874076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-7ef6875f-f62c-4d01-a29a-5d456a064779 None None] MFA Rules not processed for user `c2d55d45bd3c4ac5acff1bd098fbf472`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:02.875251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-7ef6875f-f62c-4d01-a29a-5d456a064779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:02.957428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1116/2203] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:02.979016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595be814-6691-41fe-95f3-d653bef29680 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:02.981241 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-595be814-6691-41fe-95f3-d653bef29680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:03.037642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595be814-6691-41fe-95f3-d653bef29680 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:03.041309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e5eb3-bbd4-4302-af19-d1f15b116f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:03.042510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e5eb3-bbd4-4302-af19-d1f15b116f3d service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:03.042798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e5eb3-bbd4-4302-af19-d1f15b116f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:03.043079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e5eb3-bbd4-4302-af19-d1f15b116f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:03.052070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e5eb3-bbd4-4302-af19-d1f15b116f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:03.053807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e5eb3-bbd4-4302-af19-d1f15b116f3d service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:03.064603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1088/2204] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:03.293101 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6875f-f62c-4d01-a29a-5d456a064779 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:03.303479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6875f-f62c-4d01-a29a-5d456a064779 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:03.307807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d411772-f7cc-4031-ad31-cd2085431035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:03.308937 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411772-f7cc-4031-ad31-cd2085431035 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:03.308937 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411772-f7cc-4031-ad31-cd2085431035 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:03.309337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411772-f7cc-4031-ad31-cd2085431035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:03.369159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d411772-f7cc-4031-ad31-cd2085431035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:03.370693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d411772-f7cc-4031-ad31-cd2085431035 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:03.379960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1117/2205] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:03.932401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:04.726412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:06.299001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:06.510255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:06.926852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e5eb3-bbd4-4302-af19-d1f15b116f3d service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:06.999332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e5eb3-bbd4-4302-af19-d1f15b116f3d service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:07.003927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:07.004865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:07.005150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:07.005446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] PATH_INFO: `/v3/users/c0e59127f52f415ea312d2b1a50b3762` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:07.006642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e59127f52f415ea312d2b1a50b3762)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:07.008574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:07.041400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:07.184502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:07.230258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e59127f52f415ea312d2b1a50b3762'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:07.231007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e59127f52f415ea312d2b1a50b3762'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:07.247107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e59127f52f415ea312d2b1a50b3762'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:07.255033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e59127f52f415ea312d2b1a50b3762'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:07.336259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e59127f52f415ea312d2b1a50b3762'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:07.343775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1089/2206] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:00:06 2022] DELETE /identity/v3/users/c0e59127f52f415ea312d2b1a50b3762 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:07.348452 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868527af-812f-47e7-af0a-81d947382a42 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:07.348736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868527af-812f-47e7-af0a-81d947382a42 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:07.349014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868527af-812f-47e7-af0a-81d947382a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:07.368602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-868527af-812f-47e7-af0a-81d947382a42 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:07.369857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-868527af-812f-47e7-af0a-81d947382a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:07.435530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1118/2207] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:07.445368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:07.446941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:07.508364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050acfc8-3a6a-4f7c-a7fb-e1a332dea003 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:07.512652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6a24b-e85e-423c-881f-bd717d261991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:07.513406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6a24b-e85e-423c-881f-bd717d261991 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:07.513704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6a24b-e85e-423c-881f-bd717d261991 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:07.514069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6a24b-e85e-423c-881f-bd717d261991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:07.523475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6a24b-e85e-423c-881f-bd717d261991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:07.524922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6a24b-e85e-423c-881f-bd717d261991 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:07.534568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1090/2208] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:07.664644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:07.841763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:08.016371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868527af-812f-47e7-af0a-81d947382a42 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:08.018942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-868527af-812f-47e7-af0a-81d947382a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:08.056729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868527af-812f-47e7-af0a-81d947382a42 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:08.065010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:08.066191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:08.066485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:08.066767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] PATH_INFO: `/v3/projects/55e5440700564161b28cd3e21d7d64d3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:08.069826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e5440700564161b28cd3e21d7d64d3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:08.070769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:08.267002 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e5440700564161b28cd3e21d7d64d3', 'request_id': 'req-b6201f64-09db-421c-86bc-2942fc028c32'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.272364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e5440700564161b28cd3e21d7d64d3', 'request_id': 'req-b6201f64-09db-421c-86bc-2942fc028c32'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.286873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:08.395481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e5440700564161b28cd3e21d7d64d3', 'request_id': 'req-b6201f64-09db-421c-86bc-2942fc028c32'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.408371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] The token cache is being invalidate because project 55e5440700564161b28cd3e21d7d64d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:08.408787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.409518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.411548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1119/2209] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:08 2022] DELETE /identity/v3/projects/55e5440700564161b28cd3e21d7d64d3 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:08.415598 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6a24b-e85e-423c-881f-bd717d261991 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:08.426235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6a24b-e85e-423c-881f-bd717d261991 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:08.493702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:INSERT=5,keystone:DELETE=11 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:08.502803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:08.502803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:08.502803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:08.502803 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] PATH_INFO: `/v3/users/c2d55d45bd3c4ac5acff1bd098fbf472` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:08.504353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d55d45bd3c4ac5acff1bd098fbf472)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:08.506623 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:08.763057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d55d45bd3c4ac5acff1bd098fbf472'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.763915 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d55d45bd3c4ac5acff1bd098fbf472'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.778602 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d55d45bd3c4ac5acff1bd098fbf472'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.789391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d55d45bd3c4ac5acff1bd098fbf472'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.903839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f400432d-4681-411b-b5be-1b196be5b69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d55d45bd3c4ac5acff1bd098fbf472'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:08.911579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1091/2210] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:00:08 2022] DELETE /identity/v3/users/c2d55d45bd3c4ac5acff1bd098fbf472 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:00:09.270162 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:09.410583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:09.413755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:09.424392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:09.425569 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6201f64-09db-421c-86bc-2942fc028c32 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:09.494508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:09.495465 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:09.495823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:09.496107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] PATH_INFO: `/v3/projects/b294a85f88cc45e0968d55a0b15af1f4` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:09.497342 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b294a85f88cc45e0968d55a0b15af1f4)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:09.499382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:09.619557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b294a85f88cc45e0968d55a0b15af1f4', 'request_id': 'req-d96079f2-59e4-456d-b473-fd81b14567ec'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:09.625079 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b294a85f88cc45e0968d55a0b15af1f4', 'request_id': 'req-d96079f2-59e4-456d-b473-fd81b14567ec'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:09.721924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b294a85f88cc45e0968d55a0b15af1f4', 'request_id': 'req-d96079f2-59e4-456d-b473-fd81b14567ec'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:09.746422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:09.751571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] The token cache is being invalidate because project b294a85f88cc45e0968d55a0b15af1f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:09.752166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:09.752884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-d96079f2-59e4-456d-b473-fd81b14567ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:09.754769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1120/2211] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:09 2022] DELETE /identity/v3/projects/b294a85f88cc45e0968d55a0b15af1f4 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:09.759968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ef94-131e-4f25-8fb4-1c7866479112 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:09.760288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ef94-131e-4f25-8fb4-1c7866479112 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:09.760566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ef94-131e-4f25-8fb4-1c7866479112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:09.788156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3db2ef94-131e-4f25-8fb4-1c7866479112 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:09.789372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3db2ef94-131e-4f25-8fb4-1c7866479112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:09.869139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1092/2212] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:09.878354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126ea73-51e8-429c-a69c-22b5d239a342 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:09.878952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126ea73-51e8-429c-a69c-22b5d239a342 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:09.879241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126ea73-51e8-429c-a69c-22b5d239a342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:09.906945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6126ea73-51e8-429c-a69c-22b5d239a342 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:09.910860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6126ea73-51e8-429c-a69c-22b5d239a342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:09.911199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:09.920698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1121/2213] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:09.925197 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ef94-131e-4f25-8fb4-1c7866479112 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:09.936060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ef94-131e-4f25-8fb4-1c7866479112 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:09.940755 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f69b32-2cde-4878-a6c7-247c59c8d1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:09.941465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f69b32-2cde-4878-a6c7-247c59c8d1cb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:09.941710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f69b32-2cde-4878-a6c7-247c59c8d1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:09.942029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f69b32-2cde-4878-a6c7-247c59c8d1cb admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:09.947656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f69b32-2cde-4878-a6c7-247c59c8d1cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:00:09.947656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f69b32-2cde-4878-a6c7-247c59c8d1cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:09.947656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f69b32-2cde-4878-a6c7-247c59c8d1cb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:09.957228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1093/2214] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:00:09.961646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126ea73-51e8-429c-a69c-22b5d239a342 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:09.976324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126ea73-51e8-429c-a69c-22b5d239a342 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:09.976324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6eeac-ac01-49d6-af96-e5955a87e771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:09.976963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6eeac-ac01-49d6-af96-e5955a87e771 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:09.977245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6eeac-ac01-49d6-af96-e5955a87e771 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:09.977522 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6eeac-ac01-49d6-af96-e5955a87e771 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:09.978262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6eeac-ac01-49d6-af96-e5955a87e771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:09.980752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6eeac-ac01-49d6-af96-e5955a87e771 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:10.072222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1122/2215] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:09 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:10.074753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:10.075237 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f69b32-2cde-4878-a6c7-247c59c8d1cb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:10.085306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f69b32-2cde-4878-a6c7-247c59c8d1cb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:10.148966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f32867-dfc6-40d0-8f36-938df03b0e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:10.149644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f32867-dfc6-40d0-8f36-938df03b0e5d admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:10.149972 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f32867-dfc6-40d0-8f36-938df03b0e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:10.150256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f32867-dfc6-40d0-8f36-938df03b0e5d admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:10.151496 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f32867-dfc6-40d0-8f36-938df03b0e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:10.153418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f32867-dfc6-40d0-8f36-938df03b0e5d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:10.189801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:10.297442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1094/2216] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:10 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:10.301599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6eeac-ac01-49d6-af96-e5955a87e771 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:10.312392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6eeac-ac01-49d6-af96-e5955a87e771 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:10.316858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5379c81-d361-42b2-9eeb-446abaa2c7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:10.317508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5379c81-d361-42b2-9eeb-446abaa2c7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:10.317792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5379c81-d361-42b2-9eeb-446abaa2c7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:10.318094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5379c81-d361-42b2-9eeb-446abaa2c7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:10.318812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5379c81-d361-42b2-9eeb-446abaa2c7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:10.320503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5379c81-d361-42b2-9eeb-446abaa2c7c8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:10.331845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1123/2217] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:10.356449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f32867-dfc6-40d0-8f36-938df03b0e5d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:10.367001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f32867-dfc6-40d0-8f36-938df03b0e5d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:10.371736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1cd15-1491-48de-91bc-58be9654d3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:10.372391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1cd15-1491-48de-91bc-58be9654d3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:10.372678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1cd15-1491-48de-91bc-58be9654d3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:10.372960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1cd15-1491-48de-91bc-58be9654d3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:10.373710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1cd15-1491-48de-91bc-58be9654d3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:10.375353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1cd15-1491-48de-91bc-58be9654d3f6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:10.386101 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1095/2218] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:10.390321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5379c81-d361-42b2-9eeb-446abaa2c7c8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:10.400039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5379c81-d361-42b2-9eeb-446abaa2c7c8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:10.404994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218ec46-3d89-45ee-abcc-8ed9f2a4cae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:10.405864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218ec46-3d89-45ee-abcc-8ed9f2a4cae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:10.406167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218ec46-3d89-45ee-abcc-8ed9f2a4cae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:10.406446 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218ec46-3d89-45ee-abcc-8ed9f2a4cae3 admin admin] PATH_INFO: `/v3/projects/fa83faca51fc4e8fbc85384d863f515e/users/eeafd4217a1140599cd9755757bef5bf/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:10.421467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218ec46-3d89-45ee-abcc-8ed9f2a4cae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa83faca51fc4e8fbc85384d863f515e, user_id=eeafd4217a1140599cd9755757bef5bf, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:10.424224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218ec46-3d89-45ee-abcc-8ed9f2a4cae3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:10.511801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1124/2219] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:10 2022] PUT /identity/v3/projects/fa83faca51fc4e8fbc85384d863f515e/users/eeafd4217a1140599cd9755757bef5bf/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:00:10.519740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:10.523331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c824119-85c6-498e-8559-b1f83a64e2d7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:10.523832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c824119-85c6-498e-8559-b1f83a64e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:10.524038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c824119-85c6-498e-8559-b1f83a64e2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:10.554553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7c824119-85c6-498e-8559-b1f83a64e2d7 None None] MFA Rules not processed for user `eeafd4217a1140599cd9755757bef5bf`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:10.555974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7c824119-85c6-498e-8559-b1f83a64e2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:10.630740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1096/2220] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:10.634906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218ec46-3d89-45ee-abcc-8ed9f2a4cae3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:10.700750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218ec46-3d89-45ee-abcc-8ed9f2a4cae3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:10.705707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757870a-e7f3-4481-9f01-9894390f5254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:10.706458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757870a-e7f3-4481-9f01-9894390f5254 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:10.706856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757870a-e7f3-4481-9f01-9894390f5254 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:10.707188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757870a-e7f3-4481-9f01-9894390f5254 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:10.707951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757870a-e7f3-4481-9f01-9894390f5254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:10.709925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757870a-e7f3-4481-9f01-9894390f5254 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:10.732734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:10.819211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1125/2221] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:10 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:10.819346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c824119-85c6-498e-8559-b1f83a64e2d7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:10.819647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:10.826377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c824119-85c6-498e-8559-b1f83a64e2d7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:10.884842 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c9227-76a6-472d-a0a6-86da60fd3548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:10.885580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c9227-76a6-472d-a0a6-86da60fd3548 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:10.885948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c9227-76a6-472d-a0a6-86da60fd3548 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:10.886233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c9227-76a6-472d-a0a6-86da60fd3548 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:10.887190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c9227-76a6-472d-a0a6-86da60fd3548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:10.889305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c9227-76a6-472d-a0a6-86da60fd3548 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:10.984958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1097/2222] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:10 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:10.988867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757870a-e7f3-4481-9f01-9894390f5254 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:10.999218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757870a-e7f3-4481-9f01-9894390f5254 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:11.004150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34ca0e-cd77-418c-98ec-8560614eae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:11.004841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34ca0e-cd77-418c-98ec-8560614eae77 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:11.005172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34ca0e-cd77-418c-98ec-8560614eae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:11.005491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34ca0e-cd77-418c-98ec-8560614eae77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:11.006259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34ca0e-cd77-418c-98ec-8560614eae77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:11.008112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34ca0e-cd77-418c-98ec-8560614eae77 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:11.020786 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1126/2223] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:11.023432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c9227-76a6-472d-a0a6-86da60fd3548 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:11.034137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c9227-76a6-472d-a0a6-86da60fd3548 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:11.039063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307311f-409d-4c45-8fc6-382c45658370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:11.039983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307311f-409d-4c45-8fc6-382c45658370 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:11.040276 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307311f-409d-4c45-8fc6-382c45658370 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:11.040554 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307311f-409d-4c45-8fc6-382c45658370 admin admin] PATH_INFO: `/v3/projects/a333fe0d2ae7460c859fbde1eb8a9777/users/9e0e8c84422a429589f2331171523dd9/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:11.056844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307311f-409d-4c45-8fc6-382c45658370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a333fe0d2ae7460c859fbde1eb8a9777, user_id=9e0e8c84422a429589f2331171523dd9, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:11.059582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307311f-409d-4c45-8fc6-382c45658370 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:11.091085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:11.131300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1098/2224] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:11 2022] PUT /identity/v3/projects/a333fe0d2ae7460c859fbde1eb8a9777/users/9e0e8c84422a429589f2331171523dd9/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:11.132521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34ca0e-cd77-418c-98ec-8560614eae77 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:11.144581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34ca0e-cd77-418c-98ec-8560614eae77 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:11.149118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:11.228607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb5769-5dc0-4e42-b82a-d3792d4f142c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:11.229344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb5769-5dc0-4e42-b82a-d3792d4f142c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:11.229652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb5769-5dc0-4e42-b82a-d3792d4f142c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:11.230137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb5769-5dc0-4e42-b82a-d3792d4f142c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:11.231480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb5769-5dc0-4e42-b82a-d3792d4f142c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:11.233088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb5769-5dc0-4e42-b82a-d3792d4f142c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:11.243647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1127/2225] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:11.252599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307311f-409d-4c45-8fc6-382c45658370 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:11.258942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307311f-409d-4c45-8fc6-382c45658370 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:11.266165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07652717-fba6-45a3-850b-3dca557f7f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:11.267180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07652717-fba6-45a3-850b-3dca557f7f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:11.267535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07652717-fba6-45a3-850b-3dca557f7f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:11.267880 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07652717-fba6-45a3-850b-3dca557f7f1f admin admin] PATH_INFO: `/v3/projects/a333fe0d2ae7460c859fbde1eb8a9777/users/9e0e8c84422a429589f2331171523dd9/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:11.270326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07652717-fba6-45a3-850b-3dca557f7f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a333fe0d2ae7460c859fbde1eb8a9777, user_id=9e0e8c84422a429589f2331171523dd9, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:11.273132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07652717-fba6-45a3-850b-3dca557f7f1f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:11.284884 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1099/2226] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:11 2022] PUT /identity/v3/projects/a333fe0d2ae7460c859fbde1eb8a9777/users/9e0e8c84422a429589f2331171523dd9/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:11.315576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f6be-c9ee-4110-b175-fe7ee2ae7cfd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:11.316147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f6be-c9ee-4110-b175-fe7ee2ae7cfd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:11.316356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f6be-c9ee-4110-b175-fe7ee2ae7cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:11.347997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e7e4f6be-c9ee-4110-b175-fe7ee2ae7cfd None None] MFA Rules not processed for user `9e0e8c84422a429589f2331171523dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:11.349249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e7e4f6be-c9ee-4110-b175-fe7ee2ae7cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:11.388381 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:11.420911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1128/2227] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:11.432826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07652717-fba6-45a3-850b-3dca557f7f1f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:11.434601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-07652717-fba6-45a3-850b-3dca557f7f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:11.517538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07652717-fba6-45a3-850b-3dca557f7f1f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:11.522088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe3f42-9eca-42bd-b67b-cf7f376cc39e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:11.522734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe3f42-9eca-42bd-b67b-cf7f376cc39e service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:11.523037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe3f42-9eca-42bd-b67b-cf7f376cc39e service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:11.523345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe3f42-9eca-42bd-b67b-cf7f376cc39e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:11.533729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe3f42-9eca-42bd-b67b-cf7f376cc39e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:11.535202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe3f42-9eca-42bd-b67b-cf7f376cc39e service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:11.544693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1100/2228] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:11.588259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:11.693620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4f6be-c9ee-4110-b175-fe7ee2ae7cfd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:11.705658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4f6be-c9ee-4110-b175-fe7ee2ae7cfd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:11.709647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae244e14-e445-440a-b374-2a522392eba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:11.723619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae244e14-e445-440a-b374-2a522392eba4 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:11.726600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae244e14-e445-440a-b374-2a522392eba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:11.726600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae244e14-e445-440a-b374-2a522392eba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:11.794604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae244e14-e445-440a-b374-2a522392eba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:11.796324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae244e14-e445-440a-b374-2a522392eba4 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:11.806611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1129/2229] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:11.820238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:11.853987 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:12.438137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe3f42-9eca-42bd-b67b-cf7f376cc39e service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:12.529420 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe3f42-9eca-42bd-b67b-cf7f376cc39e service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:12.534323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:12.535199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:12.535485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:12.535761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] PATH_INFO: `/v3/users/eeafd4217a1140599cd9755757bef5bf` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:12.536974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeafd4217a1140599cd9755757bef5bf)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:12.539007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:12.790135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeafd4217a1140599cd9755757bef5bf'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:12.791284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeafd4217a1140599cd9755757bef5bf'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:12.806301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeafd4217a1140599cd9755757bef5bf'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:12.818859 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeafd4217a1140599cd9755757bef5bf'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:12.901546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeafd4217a1140599cd9755757bef5bf'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:12.915105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1101/2230] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:00:12 2022] DELETE /identity/v3/users/eeafd4217a1140599cd9755757bef5bf => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:12.915892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55d50c-aef0-40fd-afb9-d97e28de7126 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:12.916382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55d50c-aef0-40fd-afb9-d97e28de7126 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:12.916994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55d50c-aef0-40fd-afb9-d97e28de7126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:12.937347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fe55d50c-aef0-40fd-afb9-d97e28de7126 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:12.937347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fe55d50c-aef0-40fd-afb9-d97e28de7126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:13.004418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1130/2231] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:13.013968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:13.014444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:13.016124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:13.097520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3673f14-d0a3-4bd2-918d-7cf8bfdd18d1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:13.101590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2ed81-23c4-4c84-82e2-cbf724cf7029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:13.102374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2ed81-23c4-4c84-82e2-cbf724cf7029 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:13.102663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2ed81-23c4-4c84-82e2-cbf724cf7029 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:13.102943 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2ed81-23c4-4c84-82e2-cbf724cf7029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:13.112562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2ed81-23c4-4c84-82e2-cbf724cf7029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:13.114045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2ed81-23c4-4c84-82e2-cbf724cf7029 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:13.125329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1102/2232] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:13.337319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:13.631592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55d50c-aef0-40fd-afb9-d97e28de7126 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:13.633848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fe55d50c-aef0-40fd-afb9-d97e28de7126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:13.655255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55d50c-aef0-40fd-afb9-d97e28de7126 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:13.665236 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:13.666818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:13.667253 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:13.667676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] PATH_INFO: `/v3/projects/fa83faca51fc4e8fbc85384d863f515e` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:13.668959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa83faca51fc4e8fbc85384d863f515e)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:13.671328 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:13.856643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa83faca51fc4e8fbc85384d863f515e', 'request_id': 'req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:13.868080 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa83faca51fc4e8fbc85384d863f515e', 'request_id': 'req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:14.065819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa83faca51fc4e8fbc85384d863f515e', 'request_id': 'req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:14.164918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] The token cache is being invalidate because project fa83faca51fc4e8fbc85384d863f515e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:14.164918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:14.164918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:14.164918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1131/2233] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:13 2022] DELETE /identity/v3/projects/fa83faca51fc4e8fbc85384d863f515e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:14.168459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2ed81-23c4-4c84-82e2-cbf724cf7029 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:14.178424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2ed81-23c4-4c84-82e2-cbf724cf7029 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:14.237304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:14.238222 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:14.238510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:14.238926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] PATH_INFO: `/v3/users/9e0e8c84422a429589f2331171523dd9` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:14.240128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0e8c84422a429589f2331171523dd9)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:14.242122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:14.549828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0e8c84422a429589f2331171523dd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:14.550501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0e8c84422a429589f2331171523dd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:14.562765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0e8c84422a429589f2331171523dd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:14.570715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0e8c84422a429589f2331171523dd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:14.664054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d3733245-2fdf-4627-a16b-d5b698ea0aa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0e8c84422a429589f2331171523dd9'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:14.672167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1103/2234] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:00:14 2022] DELETE /identity/v3/users/9e0e8c84422a429589f2331171523dd9 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:14.915641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:15.320978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:15.408205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:15.419507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:15.447782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcf1c4-6b0f-4ba2-bb35-81ac7b007827 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:15.508049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:15.508905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:15.509192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:15.509473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] PATH_INFO: `/v3/projects/a333fe0d2ae7460c859fbde1eb8a9777` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:15.511305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a333fe0d2ae7460c859fbde1eb8a9777)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:15.513483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:15.654610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a333fe0d2ae7460c859fbde1eb8a9777', 'request_id': 'req-bef1f1d1-c62c-4a37-9390-fbca4dff885e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:15.660083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a333fe0d2ae7460c859fbde1eb8a9777', 'request_id': 'req-bef1f1d1-c62c-4a37-9390-fbca4dff885e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:15.772911 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a333fe0d2ae7460c859fbde1eb8a9777', 'request_id': 'req-bef1f1d1-c62c-4a37-9390-fbca4dff885e'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:15.784892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] The token cache is being invalidate because project a333fe0d2ae7460c859fbde1eb8a9777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:15.787361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:15.787361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bef1f1d1-c62c-4a37-9390-fbca4dff885e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:15.787562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1132/2235] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:15 2022] DELETE /identity/v3/projects/a333fe0d2ae7460c859fbde1eb8a9777 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:15.793239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8f498-6275-40c7-9c0c-0a0241bdc614 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:15.793557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8f498-6275-40c7-9c0c-0a0241bdc614 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:15.793838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8f498-6275-40c7-9c0c-0a0241bdc614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:15.840353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4af8f498-6275-40c7-9c0c-0a0241bdc614 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:15.841545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4af8f498-6275-40c7-9c0c-0a0241bdc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:15.937931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1104/2236] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:15.946871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895006e5-eb9c-4a98-9fac-2161014bc426 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:15.946871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895006e5-eb9c-4a98-9fac-2161014bc426 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:15.946871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895006e5-eb9c-4a98-9fac-2161014bc426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:15.969640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-895006e5-eb9c-4a98-9fac-2161014bc426 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:15.971164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-895006e5-eb9c-4a98-9fac-2161014bc426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:15.984518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1133/2237] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:15.989094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8f498-6275-40c7-9c0c-0a0241bdc614 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:16.002236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8f498-6275-40c7-9c0c-0a0241bdc614 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:16.009711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1507e-d111-4cdb-a46c-bde69698630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:16.010560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1507e-d111-4cdb-a46c-bde69698630b admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:16.010913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1507e-d111-4cdb-a46c-bde69698630b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:16.011261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1507e-d111-4cdb-a46c-bde69698630b admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:16.012353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1507e-d111-4cdb-a46c-bde69698630b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:00:16.012851 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1507e-d111-4cdb-a46c-bde69698630b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:16.017931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1507e-d111-4cdb-a46c-bde69698630b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:16.033439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1105/2238] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:00:16.039816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895006e5-eb9c-4a98-9fac-2161014bc426 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:16.049661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895006e5-eb9c-4a98-9fac-2161014bc426 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:16.055354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887738a2-9f2e-42a0-b7c0-e108a7ff87cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:16.056016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887738a2-9f2e-42a0-b7c0-e108a7ff87cd admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:16.056486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887738a2-9f2e-42a0-b7c0-e108a7ff87cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:16.057027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887738a2-9f2e-42a0-b7c0-e108a7ff87cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:16.057925 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887738a2-9f2e-42a0-b7c0-e108a7ff87cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:16.059841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887738a2-9f2e-42a0-b7c0-e108a7ff87cd admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:16.206063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1134/2239] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:16 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:16.209084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1507e-d111-4cdb-a46c-bde69698630b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:16.221860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1507e-d111-4cdb-a46c-bde69698630b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:16.279862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845a6ba-cfaa-4654-814e-6a2a70f3afad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:16.280529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845a6ba-cfaa-4654-814e-6a2a70f3afad admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:16.280809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845a6ba-cfaa-4654-814e-6a2a70f3afad admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:16.281081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845a6ba-cfaa-4654-814e-6a2a70f3afad admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:16.281753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845a6ba-cfaa-4654-814e-6a2a70f3afad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:16.283691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845a6ba-cfaa-4654-814e-6a2a70f3afad admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:16.498435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1106/2240] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:16 2022] POST /identity/v3/users => generated 291 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:16.500008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887738a2-9f2e-42a0-b7c0-e108a7ff87cd admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:16.514780 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887738a2-9f2e-42a0-b7c0-e108a7ff87cd admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:16.519991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6a6f3-8be4-4910-9b9e-1ca4342c67cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:16.520747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6a6f3-8be4-4910-9b9e-1ca4342c67cc admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:16.523515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6a6f3-8be4-4910-9b9e-1ca4342c67cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:16.523515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6a6f3-8be4-4910-9b9e-1ca4342c67cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:16.523515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6a6f3-8be4-4910-9b9e-1ca4342c67cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:16.523806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6a6f3-8be4-4910-9b9e-1ca4342c67cc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:16.540884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1135/2241] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:16.545235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845a6ba-cfaa-4654-814e-6a2a70f3afad admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:16.556571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845a6ba-cfaa-4654-814e-6a2a70f3afad admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:16.565189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a623f3-0f1b-4afc-a1c9-46227256e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:16.565963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a623f3-0f1b-4afc-a1c9-46227256e783 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:16.566314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a623f3-0f1b-4afc-a1c9-46227256e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:16.566659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a623f3-0f1b-4afc-a1c9-46227256e783 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:16.567492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a623f3-0f1b-4afc-a1c9-46227256e783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:16.569231 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a623f3-0f1b-4afc-a1c9-46227256e783 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:16.580669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1107/2242] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:16.593067 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6a6f3-8be4-4910-9b9e-1ca4342c67cc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:16.607699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6a6f3-8be4-4910-9b9e-1ca4342c67cc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:16.614846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d759366-82bd-4a2e-bf04-c2747429604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:16.616040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d759366-82bd-4a2e-bf04-c2747429604a admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:16.616509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d759366-82bd-4a2e-bf04-c2747429604a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:16.616897 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d759366-82bd-4a2e-bf04-c2747429604a admin admin] PATH_INFO: `/v3/projects/0f29cf6f628a4248ad4a2037e680f1d0/users/2a1e309e85d944339760238234a67465/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:16.633710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d759366-82bd-4a2e-bf04-c2747429604a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f29cf6f628a4248ad4a2037e680f1d0, user_id=2a1e309e85d944339760238234a67465, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:16.646327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d759366-82bd-4a2e-bf04-c2747429604a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:16.785789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1136/2243] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:16 2022] PUT /identity/v3/projects/0f29cf6f628a4248ad4a2037e680f1d0/users/2a1e309e85d944339760238234a67465/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:16.801207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356570ea-ffec-4949-8857-dfc8192fa960 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:16.801507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356570ea-ffec-4949-8857-dfc8192fa960 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:16.801793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356570ea-ffec-4949-8857-dfc8192fa960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:16.839398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-356570ea-ffec-4949-8857-dfc8192fa960 None None] MFA Rules not processed for user `2a1e309e85d944339760238234a67465`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:16.840673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-356570ea-ffec-4949-8857-dfc8192fa960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:16.912015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1108/2244] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:16.916640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d759366-82bd-4a2e-bf04-c2747429604a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:16.987747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d759366-82bd-4a2e-bf04-c2747429604a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:16.995143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d351d-c823-4f5b-8973-8d0236fe52c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:17.000280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d351d-c823-4f5b-8973-8d0236fe52c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:17.000280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d351d-c823-4f5b-8973-8d0236fe52c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:17.000280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d351d-c823-4f5b-8973-8d0236fe52c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:17.000280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d351d-c823-4f5b-8973-8d0236fe52c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:17.000280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d351d-c823-4f5b-8973-8d0236fe52c5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:17.121088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1137/2245] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:16 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:17.124877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356570ea-ffec-4949-8857-dfc8192fa960 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:17.143126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356570ea-ffec-4949-8857-dfc8192fa960 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:17.219150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455b292-7d9b-4961-b463-6517ccd81594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:17.219825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455b292-7d9b-4961-b463-6517ccd81594 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:17.220106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455b292-7d9b-4961-b463-6517ccd81594 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:17.220382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455b292-7d9b-4961-b463-6517ccd81594 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:17.221110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455b292-7d9b-4961-b463-6517ccd81594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:17.223116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455b292-7d9b-4961-b463-6517ccd81594 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:17.276000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:17.347798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1109/2246] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:17 2022] POST /identity/v3/users => generated 298 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:17.352318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d351d-c823-4f5b-8973-8d0236fe52c5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:17.364493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d351d-c823-4f5b-8973-8d0236fe52c5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:17.370899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c13f25-7ac1-44a2-b1d8-11451c17c7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:17.371971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c13f25-7ac1-44a2-b1d8-11451c17c7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:17.372260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c13f25-7ac1-44a2-b1d8-11451c17c7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:17.372544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c13f25-7ac1-44a2-b1d8-11451c17c7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:17.373462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c13f25-7ac1-44a2-b1d8-11451c17c7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:17.375412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c13f25-7ac1-44a2-b1d8-11451c17c7d2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:17.387985 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1138/2247] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:17.392183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455b292-7d9b-4961-b463-6517ccd81594 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:17.402242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455b292-7d9b-4961-b463-6517ccd81594 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:17.412492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83988b-62f3-4fd7-9d61-3e32f71958ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:17.412492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83988b-62f3-4fd7-9d61-3e32f71958ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:17.412492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83988b-62f3-4fd7-9d61-3e32f71958ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:17.412492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83988b-62f3-4fd7-9d61-3e32f71958ce admin admin] PATH_INFO: `/v3/projects/d37f9276f3194da590cb3f99e26dbf29/users/f40bfe7350b24c249cff17d677c21f44/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:17.424904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83988b-62f3-4fd7-9d61-3e32f71958ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37f9276f3194da590cb3f99e26dbf29, user_id=f40bfe7350b24c249cff17d677c21f44, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:17.427603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83988b-62f3-4fd7-9d61-3e32f71958ce admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:17.555994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1110/2248] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:17 2022] PUT /identity/v3/projects/d37f9276f3194da590cb3f99e26dbf29/users/f40bfe7350b24c249cff17d677c21f44/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:00:17.560037 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c13f25-7ac1-44a2-b1d8-11451c17c7d2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:17.570195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c13f25-7ac1-44a2-b1d8-11451c17c7d2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:17.636945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef00f5e-5456-4083-b70a-690f6534ab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:17.637630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef00f5e-5456-4083-b70a-690f6534ab0c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:17.637943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef00f5e-5456-4083-b70a-690f6534ab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:17.638224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef00f5e-5456-4083-b70a-690f6534ab0c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:17.639124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef00f5e-5456-4083-b70a-690f6534ab0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:17.640828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef00f5e-5456-4083-b70a-690f6534ab0c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:17.653412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1139/2249] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:17.657766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83988b-62f3-4fd7-9d61-3e32f71958ce admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:17.669026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83988b-62f3-4fd7-9d61-3e32f71958ce admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:17.679630 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176dab9-748a-4d4c-bb06-f283183e48a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:17.680657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176dab9-748a-4d4c-bb06-f283183e48a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:17.681033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176dab9-748a-4d4c-bb06-f283183e48a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:17.681376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176dab9-748a-4d4c-bb06-f283183e48a8 admin admin] PATH_INFO: `/v3/projects/d37f9276f3194da590cb3f99e26dbf29/users/f40bfe7350b24c249cff17d677c21f44/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:17.683933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176dab9-748a-4d4c-bb06-f283183e48a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d37f9276f3194da590cb3f99e26dbf29, user_id=f40bfe7350b24c249cff17d677c21f44, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:17.686752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176dab9-748a-4d4c-bb06-f283183e48a8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:17.697984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1111/2250] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:17 2022] PUT /identity/v3/projects/d37f9276f3194da590cb3f99e26dbf29/users/f40bfe7350b24c249cff17d677c21f44/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:17.708287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43bd88-56ed-4c70-abbf-4d388f366caa None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:17.708593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43bd88-56ed-4c70-abbf-4d388f366caa None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:17.708873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43bd88-56ed-4c70-abbf-4d388f366caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:17.737864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ae43bd88-56ed-4c70-abbf-4d388f366caa None None] MFA Rules not processed for user `f40bfe7350b24c249cff17d677c21f44`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:17.739092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ae43bd88-56ed-4c70-abbf-4d388f366caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:17.809907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1140/2251] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:17.828025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176dab9-748a-4d4c-bb06-f283183e48a8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:17.832007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8176dab9-748a-4d4c-bb06-f283183e48a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:17.890399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176dab9-748a-4d4c-bb06-f283183e48a8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:17.894207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b75fc4-3c59-452f-af01-e17e5a8183ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:17.894872 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b75fc4-3c59-452f-af01-e17e5a8183ed service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:17.895155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b75fc4-3c59-452f-af01-e17e5a8183ed service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:17.895436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b75fc4-3c59-452f-af01-e17e5a8183ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:17.956407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:17.958960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b75fc4-3c59-452f-af01-e17e5a8183ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:17.960696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b75fc4-3c59-452f-af01-e17e5a8183ed service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:17.970819 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1112/2252] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:18.851272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:19.395966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:19.742669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:20.263370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:20.797259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f675640-b417-446d-8d03-fc281dbf01e4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:20.797721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f675640-b417-446d-8d03-fc281dbf01e4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:20.797857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f675640-b417-446d-8d03-fc281dbf01e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:20.814953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1f675640-b417-446d-8d03-fc281dbf01e4 None None] MFA Rules not processed for user `6113d715500240098217caf641db763a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:20.816073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1f675640-b417-446d-8d03-fc281dbf01e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:20.872265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1141/2253] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:20.873587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:21.133137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:21.422315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:21.485780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:21.594016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:21.893139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:22.132693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:22.864835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99c95a-71cd-4a69-8a43-85a3c94f239d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:22.865135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99c95a-71cd-4a69-8a43-85a3c94f239d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:22.865467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99c95a-71cd-4a69-8a43-85a3c94f239d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:22.886855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ce99c95a-71cd-4a69-8a43-85a3c94f239d None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:22.887997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ce99c95a-71cd-4a69-8a43-85a3c94f239d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:22.960281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1113/2254] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 10:00:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:23.029401 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f675640-b417-446d-8d03-fc281dbf01e4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:23.030890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1f675640-b417-446d-8d03-fc281dbf01e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:23.074499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:23.103338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f675640-b417-446d-8d03-fc281dbf01e4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:23.108554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53412cf-6357-4c7d-8529-ae4b184770d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:23.109195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53412cf-6357-4c7d-8529-ae4b184770d1 service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:23.109963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53412cf-6357-4c7d-8529-ae4b184770d1 service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:23.109963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53412cf-6357-4c7d-8529-ae4b184770d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:23.117912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53412cf-6357-4c7d-8529-ae4b184770d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:23.120170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53412cf-6357-4c7d-8529-ae4b184770d1 service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:23.128495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1142/2255] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:00:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 03 10:00:25.929768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99c95a-71cd-4a69-8a43-85a3c94f239d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:25.943149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99c95a-71cd-4a69-8a43-85a3c94f239d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:25.949115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e045550-2314-4451-b6a6-81bc8db3c641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:25.950621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e045550-2314-4451-b6a6-81bc8db3c641 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:25.951086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e045550-2314-4451-b6a6-81bc8db3c641 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:25.951584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e045550-2314-4451-b6a6-81bc8db3c641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:25.962951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e045550-2314-4451-b6a6-81bc8db3c641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:25.964758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e045550-2314-4451-b6a6-81bc8db3c641 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:25.986767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1114/2256] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:33.160771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=16 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:34.768747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badec6c-142b-48d0-9113-86e925f06009 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:34.769328 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badec6c-142b-48d0-9113-86e925f06009 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:34.769703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badec6c-142b-48d0-9113-86e925f06009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:34.813302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2badec6c-142b-48d0-9113-86e925f06009 None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:34.815010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2badec6c-142b-48d0-9113-86e925f06009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:34.823707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1143/2257] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 10:00:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:34.845372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e045550-2314-4451-b6a6-81bc8db3c641 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:34.860989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e045550-2314-4451-b6a6-81bc8db3c641 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:34.873525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265643a-13ec-4977-a847-690a1c5a9f1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:34.874577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265643a-13ec-4977-a847-690a1c5a9f1e service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:34.875145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265643a-13ec-4977-a847-690a1c5a9f1e service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:34.875795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265643a-13ec-4977-a847-690a1c5a9f1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:34.888697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265643a-13ec-4977-a847-690a1c5a9f1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:34.890800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265643a-13ec-4977-a847-690a1c5a9f1e service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:34.902720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1115/2258] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:00:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:00:37.210858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badec6c-142b-48d0-9113-86e925f06009 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:37.328662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badec6c-142b-48d0-9113-86e925f06009 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:37.334878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:37.336008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:37.336446 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:37.336936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] PATH_INFO: `/v3/users/2a1e309e85d944339760238234a67465` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:37.338990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a1e309e85d944339760238234a67465)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:37.341395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:37.629353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a1e309e85d944339760238234a67465'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:37.630087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a1e309e85d944339760238234a67465'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:37.642355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a1e309e85d944339760238234a67465'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:37.683341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a1e309e85d944339760238234a67465'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:37.761592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a1e309e85d944339760238234a67465'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:37.792773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1144/2259] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:00:37 2022] DELETE /identity/v3/users/2a1e309e85d944339760238234a67465 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:37.794466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d4e4-7d93-4a3b-ad39-8ff64e7c2b79 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:37.795474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d4e4-7d93-4a3b-ad39-8ff64e7c2b79 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:37.796024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d4e4-7d93-4a3b-ad39-8ff64e7c2b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:37.834633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-dbe6d4e4-7d93-4a3b-ad39-8ff64e7c2b79 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:37.836780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dbe6d4e4-7d93-4a3b-ad39-8ff64e7c2b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:37.916610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:37.937208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1116/2260] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:37.947926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:37.949783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:38.160530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd42f1-c377-42dd-9ca1-835062329310 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:38.165293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193ad0e-86fb-454c-a383-c68b8775499d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:38.167376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193ad0e-86fb-454c-a383-c68b8775499d service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:38.167819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193ad0e-86fb-454c-a383-c68b8775499d service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:38.168487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193ad0e-86fb-454c-a383-c68b8775499d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:38.181062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193ad0e-86fb-454c-a383-c68b8775499d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:38.183960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193ad0e-86fb-454c-a383-c68b8775499d service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:38.198976 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:38.210795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1145/2261] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:39.166236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6d4e4-7d93-4a3b-ad39-8ff64e7c2b79 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:39.168475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dbe6d4e4-7d93-4a3b-ad39-8ff64e7c2b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:39.224090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6d4e4-7d93-4a3b-ad39-8ff64e7c2b79 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:39.248715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:39.251483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:39.251483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:39.251483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] PATH_INFO: `/v3/projects/0f29cf6f628a4248ad4a2037e680f1d0` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:39.252068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f29cf6f628a4248ad4a2037e680f1d0)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:39.266036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:39.439678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f29cf6f628a4248ad4a2037e680f1d0', 'request_id': 'req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:39.461964 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f29cf6f628a4248ad4a2037e680f1d0', 'request_id': 'req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:39.668894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f29cf6f628a4248ad4a2037e680f1d0', 'request_id': 'req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:39.697240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] The token cache is being invalidate because project 0f29cf6f628a4248ad4a2037e680f1d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:39.697798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:39.698667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:39.707684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1117/2262] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:39 2022] DELETE /identity/v3/projects/0f29cf6f628a4248ad4a2037e680f1d0 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:39.712650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193ad0e-86fb-454c-a383-c68b8775499d service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:39.737826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193ad0e-86fb-454c-a383-c68b8775499d service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:39.923839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:39.923839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:39.923839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:39.923839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] PATH_INFO: `/v3/users/f40bfe7350b24c249cff17d677c21f44` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:39.926910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f40bfe7350b24c249cff17d677c21f44)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:39.932852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:40.700968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f40bfe7350b24c249cff17d677c21f44'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:40.701619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f40bfe7350b24c249cff17d677c21f44'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:40.708793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f40bfe7350b24c249cff17d677c21f44'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:40.721859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f40bfe7350b24c249cff17d677c21f44'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:40.829500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-09bf4473-6621-4816-94fb-02d81d64fd79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f40bfe7350b24c249cff17d677c21f44'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:40.838320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1146/2263] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:00:39 2022] DELETE /identity/v3/users/f40bfe7350b24c249cff17d677c21f44 => generated 0 bytes in 1126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:41.522484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:41.527532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:41.572990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564b7ec-f4e4-4262-a134-9c9de7f1e23b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:41.707891 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:41.711227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:41.714476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:41.716197 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] PATH_INFO: `/v3/projects/d37f9276f3194da590cb3f99e26dbf29` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:41.719187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d37f9276f3194da590cb3f99e26dbf29)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:41.726910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:41.903809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd37f9276f3194da590cb3f99e26dbf29', 'request_id': 'req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:41.935760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd37f9276f3194da590cb3f99e26dbf29', 'request_id': 'req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:42.029849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd37f9276f3194da590cb3f99e26dbf29', 'request_id': 'req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:42.080692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] The token cache is being invalidate because project d37f9276f3194da590cb3f99e26dbf29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:42.083076 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:42.085500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-fadc5804-2ab1-4e5e-9f2d-c94c46c58f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:42.113282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1118/2264] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:41 2022] DELETE /identity/v3/projects/d37f9276f3194da590cb3f99e26dbf29 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:42.114863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f5af6-f583-41d3-ac41-c21f6567bc3f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:42.115316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f5af6-f583-41d3-ac41-c21f6567bc3f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:42.115466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f5af6-f583-41d3-ac41-c21f6567bc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:42.182999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ca0f5af6-f583-41d3-ac41-c21f6567bc3f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:42.184575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ca0f5af6-f583-41d3-ac41-c21f6567bc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:42.292910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1147/2265] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:42.295867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdff3f-5de1-4ab0-95a4-5492a9a8a51d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:42.296479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdff3f-5de1-4ab0-95a4-5492a9a8a51d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:42.297574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdff3f-5de1-4ab0-95a4-5492a9a8a51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:42.341432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c4cdff3f-5de1-4ab0-95a4-5492a9a8a51d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:42.343057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c4cdff3f-5de1-4ab0-95a4-5492a9a8a51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:42.361129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1119/2266] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:42.362204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f5af6-f583-41d3-ac41-c21f6567bc3f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:42.374267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f5af6-f583-41d3-ac41-c21f6567bc3f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:42.388345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09cdf2-a814-40f8-a74b-e978ddddd47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:42.389140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09cdf2-a814-40f8-a74b-e978ddddd47e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:42.390533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09cdf2-a814-40f8-a74b-e978ddddd47e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:42.402691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09cdf2-a814-40f8-a74b-e978ddddd47e admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:42.412235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09cdf2-a814-40f8-a74b-e978ddddd47e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:00:42.412984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09cdf2-a814-40f8-a74b-e978ddddd47e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:42.414502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09cdf2-a814-40f8-a74b-e978ddddd47e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:42.417827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:42.437724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1148/2267] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:00:42.438983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdff3f-5de1-4ab0-95a4-5492a9a8a51d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:42.454827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdff3f-5de1-4ab0-95a4-5492a9a8a51d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:42.463155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c747f-5da3-4224-9886-3fd28daf8824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:42.465693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c747f-5da3-4224-9886-3fd28daf8824 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:42.465693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c747f-5da3-4224-9886-3fd28daf8824 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:42.465693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c747f-5da3-4224-9886-3fd28daf8824 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:42.465693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c747f-5da3-4224-9886-3fd28daf8824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:42.474679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c747f-5da3-4224-9886-3fd28daf8824 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:42.478532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:42.594176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1120/2268] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:42 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:42.594881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09cdf2-a814-40f8-a74b-e978ddddd47e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:42.607514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09cdf2-a814-40f8-a74b-e978ddddd47e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:42.733060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e756a-f25d-4413-9948-0b54dd76f51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:42.733938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e756a-f25d-4413-9948-0b54dd76f51b admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:42.734606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e756a-f25d-4413-9948-0b54dd76f51b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:42.734606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e756a-f25d-4413-9948-0b54dd76f51b admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:42.735514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e756a-f25d-4413-9948-0b54dd76f51b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:42.737597 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e756a-f25d-4413-9948-0b54dd76f51b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:42.844032 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1149/2269] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:42 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:42.848318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c747f-5da3-4224-9886-3fd28daf8824 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:42.876378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c747f-5da3-4224-9886-3fd28daf8824 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:42.904816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50530264-102d-447e-934b-dc83c240f401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:42.908897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50530264-102d-447e-934b-dc83c240f401 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:42.911355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50530264-102d-447e-934b-dc83c240f401 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:42.911815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50530264-102d-447e-934b-dc83c240f401 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:42.927220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50530264-102d-447e-934b-dc83c240f401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:42.934419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50530264-102d-447e-934b-dc83c240f401 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:42.957959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1121/2270] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:42.960329 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e756a-f25d-4413-9948-0b54dd76f51b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:42.974221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e756a-f25d-4413-9948-0b54dd76f51b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:42.991807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7583435-6fa0-4e9b-92f7-fe60d62ee0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:42.992957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7583435-6fa0-4e9b-92f7-fe60d62ee0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:42.993412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7583435-6fa0-4e9b-92f7-fe60d62ee0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:42.994518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7583435-6fa0-4e9b-92f7-fe60d62ee0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:42.995470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7583435-6fa0-4e9b-92f7-fe60d62ee0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:42.997304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7583435-6fa0-4e9b-92f7-fe60d62ee0a0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:43.023803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1150/2271] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:43.029279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50530264-102d-447e-934b-dc83c240f401 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:43.043775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50530264-102d-447e-934b-dc83c240f401 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:43.060297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba83ac-552f-4abe-b701-3152c715aca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:43.061425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba83ac-552f-4abe-b701-3152c715aca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:43.063442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba83ac-552f-4abe-b701-3152c715aca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:43.063869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba83ac-552f-4abe-b701-3152c715aca1 admin admin] PATH_INFO: `/v3/projects/f42bc72bafb546caada0101e26e91051/users/0398c0073372409a82d83fa84bf80506/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:43.098179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba83ac-552f-4abe-b701-3152c715aca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42bc72bafb546caada0101e26e91051, user_id=0398c0073372409a82d83fa84bf80506, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:43.102130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba83ac-552f-4abe-b701-3152c715aca1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:43.200644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1122/2272] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:43 2022] PUT /identity/v3/projects/f42bc72bafb546caada0101e26e91051/users/0398c0073372409a82d83fa84bf80506/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:43.207690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c624-6848-4717-a7dc-96cb2bb93333 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:43.208261 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c624-6848-4717-a7dc-96cb2bb93333 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:43.208347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c624-6848-4717-a7dc-96cb2bb93333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:43.289011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4b92c624-6848-4717-a7dc-96cb2bb93333 None None] MFA Rules not processed for user `0398c0073372409a82d83fa84bf80506`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:43.290668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4b92c624-6848-4717-a7dc-96cb2bb93333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:43.408118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1151/2273] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:43.422377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba83ac-552f-4abe-b701-3152c715aca1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:43.425528 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-84ba83ac-552f-4abe-b701-3152c715aca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:43.615791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba83ac-552f-4abe-b701-3152c715aca1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:43.621060 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cac5f-2028-48f2-9feb-ffaaf4ade00f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:43.621789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cac5f-2028-48f2-9feb-ffaaf4ade00f service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:43.622135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cac5f-2028-48f2-9feb-ffaaf4ade00f service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:43.622419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cac5f-2028-48f2-9feb-ffaaf4ade00f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:43.645152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cac5f-2028-48f2-9feb-ffaaf4ade00f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:43.646658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cac5f-2028-48f2-9feb-ffaaf4ade00f service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:43.682801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1123/2274] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:43.735594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92c624-6848-4717-a7dc-96cb2bb93333 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:43.966780 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92c624-6848-4717-a7dc-96cb2bb93333 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:43.976996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:43.979145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:43.979640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:43.979918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] PATH_INFO: `/v3/users/0398c0073372409a82d83fa84bf80506` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:43.985858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0398c0073372409a82d83fa84bf80506)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:43.988320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:44.018151 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:44.288884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0398c0073372409a82d83fa84bf80506'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:44.291573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0398c0073372409a82d83fa84bf80506'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:44.322922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0398c0073372409a82d83fa84bf80506'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:44.350695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0398c0073372409a82d83fa84bf80506'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:44.806796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0398c0073372409a82d83fa84bf80506'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:44.806796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1152/2275] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:00:43 2022] DELETE /identity/v3/users/0398c0073372409a82d83fa84bf80506 => generated 0 bytes in 1067 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:44.808801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c634b5d-b39e-4b4d-95c0-7682be3821ca None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:44.809367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c634b5d-b39e-4b4d-95c0-7682be3821ca None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:44.809367 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c634b5d-b39e-4b4d-95c0-7682be3821ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:44.826780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2c634b5d-b39e-4b4d-95c0-7682be3821ca None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:44.828116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2c634b5d-b39e-4b4d-95c0-7682be3821ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:44.909361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1124/2276] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:44.924885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:44.926502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:44.989341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b69f3-8ca8-41c8-a9e9-18651c303aa0 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:44.994484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95479ac-cdd9-4df9-9908-8d47916a2ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:44.995147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95479ac-cdd9-4df9-9908-8d47916a2ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:44.995426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95479ac-cdd9-4df9-9908-8d47916a2ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:44.995707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95479ac-cdd9-4df9-9908-8d47916a2ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:45.006013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95479ac-cdd9-4df9-9908-8d47916a2ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:45.007443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95479ac-cdd9-4df9-9908-8d47916a2ae3 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:45.017531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1153/2277] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:45.931484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c634b5d-b39e-4b4d-95c0-7682be3821ca None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:45.934264 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2c634b5d-b39e-4b4d-95c0-7682be3821ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:45.956278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c634b5d-b39e-4b4d-95c0-7682be3821ca None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:45.967030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:45.971559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:45.972362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:45.976196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] PATH_INFO: `/v3/projects/f42bc72bafb546caada0101e26e91051` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:45.980581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42bc72bafb546caada0101e26e91051)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:45.986994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:46.190546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:46.194989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42bc72bafb546caada0101e26e91051', 'request_id': 'req-90553123-eafc-4d77-9e04-36fbf34b6f8b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:46.214398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42bc72bafb546caada0101e26e91051', 'request_id': 'req-90553123-eafc-4d77-9e04-36fbf34b6f8b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:46.332939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42bc72bafb546caada0101e26e91051', 'request_id': 'req-90553123-eafc-4d77-9e04-36fbf34b6f8b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:46.354596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] The token cache is being invalidate because project f42bc72bafb546caada0101e26e91051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:46.358006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:46.358868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-90553123-eafc-4d77-9e04-36fbf34b6f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:46.371250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1125/2278] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:45 2022] DELETE /identity/v3/projects/f42bc72bafb546caada0101e26e91051 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:46.373013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b6935-7dc8-4010-94fd-eeafc10237e2 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:46.373511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b6935-7dc8-4010-94fd-eeafc10237e2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:46.373599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b6935-7dc8-4010-94fd-eeafc10237e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:46.407050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-979b6935-7dc8-4010-94fd-eeafc10237e2 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:46.410670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-979b6935-7dc8-4010-94fd-eeafc10237e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:46.545083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1154/2279] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:46.553774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f540ccd-ea94-48a9-95ce-35dc0734d8de None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:46.554147 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f540ccd-ea94-48a9-95ce-35dc0734d8de None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:46.554542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f540ccd-ea94-48a9-95ce-35dc0734d8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:46.591007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4f540ccd-ea94-48a9-95ce-35dc0734d8de None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:46.592251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4f540ccd-ea94-48a9-95ce-35dc0734d8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:46.611849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1126/2280] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:46.623511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b6935-7dc8-4010-94fd-eeafc10237e2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:46.642082 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b6935-7dc8-4010-94fd-eeafc10237e2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:46.647853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb0b06-79c7-4404-811e-f1e448583718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:46.648434 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb0b06-79c7-4404-811e-f1e448583718 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:46.648777 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb0b06-79c7-4404-811e-f1e448583718 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:46.649055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb0b06-79c7-4404-811e-f1e448583718 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:46.649921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb0b06-79c7-4404-811e-f1e448583718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:00:46.650270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb0b06-79c7-4404-811e-f1e448583718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:46.652087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb0b06-79c7-4404-811e-f1e448583718 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:46.694793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1155/2281] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:00:46.698261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f540ccd-ea94-48a9-95ce-35dc0734d8de None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:46.711383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f540ccd-ea94-48a9-95ce-35dc0734d8de None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:46.731976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a038177-bc05-41de-96ee-4e66de499046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:46.732658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038177-bc05-41de-96ee-4e66de499046 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:46.732945 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038177-bc05-41de-96ee-4e66de499046 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:46.733225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038177-bc05-41de-96ee-4e66de499046 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:46.733960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a038177-bc05-41de-96ee-4e66de499046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:46.735942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a038177-bc05-41de-96ee-4e66de499046 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:46.865287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1127/2282] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:46 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:46.868395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb0b06-79c7-4404-811e-f1e448583718 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:46.888521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb0b06-79c7-4404-811e-f1e448583718 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:46.908622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:47.060527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83cfce-a33e-4506-be5c-338e89c5ae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:47.061222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83cfce-a33e-4506-be5c-338e89c5ae21 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:47.061508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83cfce-a33e-4506-be5c-338e89c5ae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:47.061968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83cfce-a33e-4506-be5c-338e89c5ae21 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:47.062939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83cfce-a33e-4506-be5c-338e89c5ae21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:47.065078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83cfce-a33e-4506-be5c-338e89c5ae21 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:47.168881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1156/2283] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:46 2022] POST /identity/v3/users => generated 291 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:47.173039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a038177-bc05-41de-96ee-4e66de499046 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:47.186975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a038177-bc05-41de-96ee-4e66de499046 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:47.193006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166cf9e-f3e5-4397-9c4e-446fff35bd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:47.193877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166cf9e-f3e5-4397-9c4e-446fff35bd4a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:47.194959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166cf9e-f3e5-4397-9c4e-446fff35bd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:47.194959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b166cf9e-f3e5-4397-9c4e-446fff35bd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:47.195759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166cf9e-f3e5-4397-9c4e-446fff35bd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:47.198063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b166cf9e-f3e5-4397-9c4e-446fff35bd4a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:47.216492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1128/2284] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:47.220362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83cfce-a33e-4506-be5c-338e89c5ae21 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:47.236049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83cfce-a33e-4506-be5c-338e89c5ae21 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:47.242991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d12a5-e62d-4e63-b466-4c46bd787e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:47.244706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d12a5-e62d-4e63-b466-4c46bd787e81 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:47.245228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d12a5-e62d-4e63-b466-4c46bd787e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:47.245928 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d12a5-e62d-4e63-b466-4c46bd787e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:47.247016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d12a5-e62d-4e63-b466-4c46bd787e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:47.249089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d12a5-e62d-4e63-b466-4c46bd787e81 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:47.276044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1157/2285] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:47.281053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166cf9e-f3e5-4397-9c4e-446fff35bd4a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:47.295239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b166cf9e-f3e5-4397-9c4e-446fff35bd4a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:47.304543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280037c-ce99-4694-868e-5998ba68b3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:47.305696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280037c-ce99-4694-868e-5998ba68b3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:47.306262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280037c-ce99-4694-868e-5998ba68b3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:47.306920 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280037c-ce99-4694-868e-5998ba68b3d5 admin admin] PATH_INFO: `/v3/projects/6a0f8d737269433bb6ce1baf51b8157b/users/82621df465a640b28eb955733ceda02f/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:47.335214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280037c-ce99-4694-868e-5998ba68b3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a0f8d737269433bb6ce1baf51b8157b, user_id=82621df465a640b28eb955733ceda02f, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:47.335214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280037c-ce99-4694-868e-5998ba68b3d5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:47.417498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1129/2286] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:47 2022] PUT /identity/v3/projects/6a0f8d737269433bb6ce1baf51b8157b/users/82621df465a640b28eb955733ceda02f/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:47.423595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd940b-bc0a-44e9-8220-ef0b24657136 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:47.424682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd940b-bc0a-44e9-8220-ef0b24657136 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:47.425227 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd940b-bc0a-44e9-8220-ef0b24657136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:47.462266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-63bd940b-bc0a-44e9-8220-ef0b24657136 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:47.470356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-63bd940b-bc0a-44e9-8220-ef0b24657136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:47.685858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1158/2287] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:47.695321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280037c-ce99-4694-868e-5998ba68b3d5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:47.697736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8280037c-ce99-4694-868e-5998ba68b3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:47.871374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280037c-ce99-4694-868e-5998ba68b3d5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:47.876943 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf95ed-a71c-46c6-9e08-42a48c252d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:47.881068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf95ed-a71c-46c6-9e08-42a48c252d15 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:47.881442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf95ed-a71c-46c6-9e08-42a48c252d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:47.881796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf95ed-a71c-46c6-9e08-42a48c252d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:47.900920 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf95ed-a71c-46c6-9e08-42a48c252d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:47.902727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf95ed-a71c-46c6-9e08-42a48c252d15 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:47.932782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1130/2288] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:50.829438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fabde-c277-46b8-8f90-a9f0b88ed313 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:50.836565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fabde-c277-46b8-8f90-a9f0b88ed313 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:50.836565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fabde-c277-46b8-8f90-a9f0b88ed313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:50.899388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0c3fabde-c277-46b8-8f90-a9f0b88ed313 None None] MFA Rules not processed for user `82621df465a640b28eb955733ceda02f`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:50.902509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0c3fabde-c277-46b8-8f90-a9f0b88ed313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:51.062343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1159/2289] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:51.070015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf95ed-a71c-46c6-9e08-42a48c252d15 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:51.072815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9ecf95ed-a71c-46c6-9e08-42a48c252d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:51.222227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf95ed-a71c-46c6-9e08-42a48c252d15 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:51.226125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dab185-144d-4eda-95ba-b6619a4f27be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:51.226786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dab185-144d-4eda-95ba-b6619a4f27be service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:51.227151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dab185-144d-4eda-95ba-b6619a4f27be service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:51.227508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dab185-144d-4eda-95ba-b6619a4f27be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:51.238725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dab185-144d-4eda-95ba-b6619a4f27be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:51.242835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dab185-144d-4eda-95ba-b6619a4f27be service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:51.244789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:51.252587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1131/2290] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:51.363916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fabde-c277-46b8-8f90-a9f0b88ed313 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:51.365398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0c3fabde-c277-46b8-8f90-a9f0b88ed313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:51.381567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fabde-c277-46b8-8f90-a9f0b88ed313 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:51.391501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:51.391501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:51.392110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:51.392501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] PATH_INFO: `/v3/users/c7c6f34b37b2424cad0aa958ed88b2b8` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:51.394104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c6f34b37b2424cad0aa958ed88b2b8)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:51.396960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:51.417582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:51.836790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c6f34b37b2424cad0aa958ed88b2b8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:51.838046 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c6f34b37b2424cad0aa958ed88b2b8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:51.844699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c6f34b37b2424cad0aa958ed88b2b8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:51.856044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c6f34b37b2424cad0aa958ed88b2b8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:51.993028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c6f34b37b2424cad0aa958ed88b2b8'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:52.002768 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1160/2291] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:00:51 2022] DELETE /identity/v3/users/c7c6f34b37b2424cad0aa958ed88b2b8 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:52.004440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e44b7-11e7-48bb-9d52-72d96b0b24e8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:52.004733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e44b7-11e7-48bb-9d52-72d96b0b24e8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:52.005012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e44b7-11e7-48bb-9d52-72d96b0b24e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:52.026198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-906e44b7-11e7-48bb-9d52-72d96b0b24e8 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:52.027658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-906e44b7-11e7-48bb-9d52-72d96b0b24e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:52.087241 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1132/2292] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:52.097473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:52.099095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:52.156916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa468b28-2f40-4d6a-94dd-d5a0b448dd70 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:52.160562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299303-9c8f-430c-a40d-5ae0706f9615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:52.161211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299303-9c8f-430c-a40d-5ae0706f9615 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:52.161582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299303-9c8f-430c-a40d-5ae0706f9615 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:52.161757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299303-9c8f-430c-a40d-5ae0706f9615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:52.171357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8299303-9c8f-430c-a40d-5ae0706f9615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:52.172805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8299303-9c8f-430c-a40d-5ae0706f9615 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:52.183144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1161/2293] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:52.755145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e44b7-11e7-48bb-9d52-72d96b0b24e8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:52.756550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-906e44b7-11e7-48bb-9d52-72d96b0b24e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:52.769751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e44b7-11e7-48bb-9d52-72d96b0b24e8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:52.774453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:52.776822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:52.777109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:52.777385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] PATH_INFO: `/v3/projects/4718707db7fa48d3a4e711e0818fd861` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:52.778583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4718707db7fa48d3a4e711e0818fd861)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:52.780604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:52.898257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4718707db7fa48d3a4e711e0818fd861', 'request_id': 'req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:52.904244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4718707db7fa48d3a4e711e0818fd861', 'request_id': 'req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:53.023802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4718707db7fa48d3a4e711e0818fd861', 'request_id': 'req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:53.035790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] The token cache is being invalidate because project 4718707db7fa48d3a4e711e0818fd861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:53.039731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:53.039731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-842d24c8-88b3-46bb-bf4f-3e0dc3c8cb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:53.039731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1133/2294] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:52 2022] DELETE /identity/v3/projects/4718707db7fa48d3a4e711e0818fd861 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:53.043622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503828d5-dc32-473f-8ede-0073f7648b34 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.044075 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503828d5-dc32-473f-8ede-0073f7648b34 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.044201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503828d5-dc32-473f-8ede-0073f7648b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.062216 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-503828d5-dc32-473f-8ede-0073f7648b34 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:53.063402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-503828d5-dc32-473f-8ede-0073f7648b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:53.131298 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1162/2295] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:53.140698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a57b2-2389-42eb-ba91-f0df38ffec4b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.141223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a57b2-2389-42eb-ba91-f0df38ffec4b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.141908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a57b2-2389-42eb-ba91-f0df38ffec4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.161413 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ce0a57b2-2389-42eb-ba91-f0df38ffec4b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:53.162658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ce0a57b2-2389-42eb-ba91-f0df38ffec4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:53.170970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1134/2296] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:53.175378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503828d5-dc32-473f-8ede-0073f7648b34 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:53.185632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503828d5-dc32-473f-8ede-0073f7648b34 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:53.190337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8e63d-4e1a-4bce-86b6-450eb5e023db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:53.190999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8e63d-4e1a-4bce-86b6-450eb5e023db admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.191287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8e63d-4e1a-4bce-86b6-450eb5e023db admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.191566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8e63d-4e1a-4bce-86b6-450eb5e023db admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.192295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8e63d-4e1a-4bce-86b6-450eb5e023db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:00:53.192605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8e63d-4e1a-4bce-86b6-450eb5e023db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:53.194251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8e63d-4e1a-4bce-86b6-450eb5e023db admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:53.209334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1163/2297] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:00:53.211374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a57b2-2389-42eb-ba91-f0df38ffec4b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:53.221282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a57b2-2389-42eb-ba91-f0df38ffec4b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:53.225995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47ee18-d033-40fd-b507-4c22f6b6f0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:53.226682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47ee18-d033-40fd-b507-4c22f6b6f0fb admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.226968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47ee18-d033-40fd-b507-4c22f6b6f0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.227254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47ee18-d033-40fd-b507-4c22f6b6f0fb admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.227941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47ee18-d033-40fd-b507-4c22f6b6f0fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:53.229908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47ee18-d033-40fd-b507-4c22f6b6f0fb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:53.362037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1135/2298] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:53 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:00:53.366299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8e63d-4e1a-4bce-86b6-450eb5e023db admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:53.376314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8e63d-4e1a-4bce-86b6-450eb5e023db admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:53.424813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:53.467991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fc68f-7d7f-4b2d-abb7-d59f06925629 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.472333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fc68f-7d7f-4b2d-abb7-d59f06925629 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.472333 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fc68f-7d7f-4b2d-abb7-d59f06925629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.479495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53c16-ca1d-4185-b34a-a1668c590f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:53.480315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53c16-ca1d-4185-b34a-a1668c590f34 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.480700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53c16-ca1d-4185-b34a-a1668c590f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.481082 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53c16-ca1d-4185-b34a-a1668c590f34 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.481876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc53c16-ca1d-4185-b34a-a1668c590f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:53.484867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc53c16-ca1d-4185-b34a-a1668c590f34 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:53.488143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f17fc68f-7d7f-4b2d-abb7-d59f06925629 None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:53.489351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f17fc68f-7d7f-4b2d-abb7-d59f06925629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:53.540666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:53.557594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1136/2299] 10.222.0.42 () {58 vars in 967 bytes} [Fri Jun 3 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:53.571577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fc68f-7d7f-4b2d-abb7-d59f06925629 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:53.574152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f17fc68f-7d7f-4b2d-abb7-d59f06925629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:53.620048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1164/2300] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:53 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:53.624792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53c16-ca1d-4185-b34a-a1668c590f34 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:53.635440 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53c16-ca1d-4185-b34a-a1668c590f34 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:53.640741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f852c19-2224-40b7-ae38-9e20d870d9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:53.641525 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f852c19-2224-40b7-ae38-9e20d870d9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.641915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f852c19-2224-40b7-ae38-9e20d870d9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.642269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f852c19-2224-40b7-ae38-9e20d870d9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.643208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f852c19-2224-40b7-ae38-9e20d870d9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:53.645120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f852c19-2224-40b7-ae38-9e20d870d9d4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:53.650518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fc68f-7d7f-4b2d-abb7-d59f06925629 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:53.656638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1165/2301] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:53.657498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d6292-799c-4c93-8e09-4cd3c1e7e90b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:53.658263 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d6292-799c-4c93-8e09-4cd3c1e7e90b service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.658603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d6292-799c-4c93-8e09-4cd3c1e7e90b service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.658940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d6292-799c-4c93-8e09-4cd3c1e7e90b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.660789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f852c19-2224-40b7-ae38-9e20d870d9d4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:53.674342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d6292-799c-4c93-8e09-4cd3c1e7e90b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:53.687618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d6292-799c-4c93-8e09-4cd3c1e7e90b service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:53.687758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f852c19-2224-40b7-ae38-9e20d870d9d4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:53.688391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1137/2302] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:00:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:00:53.689377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9298d-574f-428b-ac5c-949004dcf0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:53.690600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9298d-574f-428b-ac5c-949004dcf0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.691036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9298d-574f-428b-ac5c-949004dcf0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.691364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9298d-574f-428b-ac5c-949004dcf0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.692042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9298d-574f-428b-ac5c-949004dcf0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:53.694337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9298d-574f-428b-ac5c-949004dcf0d9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:53.706594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1166/2303] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:00:53.714063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d6292-799c-4c93-8e09-4cd3c1e7e90b service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:53.722226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d6292-799c-4c93-8e09-4cd3c1e7e90b service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:53.726983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20519d2-10cf-4674-be8e-3f28a0059887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:53.734154 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20519d2-10cf-4674-be8e-3f28a0059887 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.734154 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20519d2-10cf-4674-be8e-3f28a0059887 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.734154 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20519d2-10cf-4674-be8e-3f28a0059887 admin admin] PATH_INFO: `/v3/projects/244fa4e99f7348629c9b5506f21cf21e/users/f6830a0ff301407a89c88429241b2379/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.742826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20519d2-10cf-4674-be8e-3f28a0059887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244fa4e99f7348629c9b5506f21cf21e, user_id=f6830a0ff301407a89c88429241b2379, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:53.746436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20519d2-10cf-4674-be8e-3f28a0059887 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:53.888099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1138/2304] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:53 2022] PUT /identity/v3/projects/244fa4e99f7348629c9b5506f21cf21e/users/f6830a0ff301407a89c88429241b2379/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:53.898418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec9a02-db0f-4693-afda-79271ab1f505 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:53.898734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec9a02-db0f-4693-afda-79271ab1f505 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:53.899020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec9a02-db0f-4693-afda-79271ab1f505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:53.953411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-51ec9a02-db0f-4693-afda-79271ab1f505 None None] MFA Rules not processed for user `f6830a0ff301407a89c88429241b2379`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:53.953411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-51ec9a02-db0f-4693-afda-79271ab1f505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:53.989592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1167/2305] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:54.000951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20519d2-10cf-4674-be8e-3f28a0059887 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:54.002755 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e20519d2-10cf-4674-be8e-3f28a0059887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:54.059162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20519d2-10cf-4674-be8e-3f28a0059887 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:54.062885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1e72a-944e-45e3-8ecd-37d2fb23fee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:54.063526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1e72a-944e-45e3-8ecd-37d2fb23fee5 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:54.063814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1e72a-944e-45e3-8ecd-37d2fb23fee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:54.064095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1e72a-944e-45e3-8ecd-37d2fb23fee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:54.072537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1e72a-944e-45e3-8ecd-37d2fb23fee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:54.074002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a1e72a-944e-45e3-8ecd-37d2fb23fee5 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:54.082700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1139/2306] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:54.491051 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610f4a4-32dd-4b89-8497-f649e557231d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:54.495950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610f4a4-32dd-4b89-8497-f649e557231d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:54.495950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610f4a4-32dd-4b89-8497-f649e557231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:54.509670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1610f4a4-32dd-4b89-8497-f649e557231d None None] MFA Rules not processed for user `82621df465a640b28eb955733ceda02f`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:54.510897 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1610f4a4-32dd-4b89-8497-f649e557231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:54.567270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1168/2307] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:55.956771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c50b1-ff88-449f-969c-4192885de8bc None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:55.957677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c50b1-ff88-449f-969c-4192885de8bc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:55.957677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c50b1-ff88-449f-969c-4192885de8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:55.984614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a13c50b1-ff88-449f-969c-4192885de8bc None None] MFA Rules not processed for user `f6830a0ff301407a89c88429241b2379`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:55.985869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a13c50b1-ff88-449f-969c-4192885de8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:55.994507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1140/2308] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:57.871037 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610f4a4-32dd-4b89-8497-f649e557231d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:57.872733 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1610f4a4-32dd-4b89-8497-f649e557231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:57.906097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:57.941214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610f4a4-32dd-4b89-8497-f649e557231d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:57.945946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:57.949750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:57.949750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:57.949750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] PATH_INFO: `/v3/users/3dd300436c9545faa56d60ac290bc987` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:57.949750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd300436c9545faa56d60ac290bc987)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:57.950568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:58.238662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd300436c9545faa56d60ac290bc987'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:58.242640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd300436c9545faa56d60ac290bc987'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:58.255588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd300436c9545faa56d60ac290bc987'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:58.274448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd300436c9545faa56d60ac290bc987'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:58.426139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd300436c9545faa56d60ac290bc987'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:58.437419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8faaf-6716-4f6c-af5c-e0665a7c3176 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:58.438883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8faaf-6716-4f6c-af5c-e0665a7c3176 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:58.439496 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8faaf-6716-4f6c-af5c-e0665a7c3176 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:58.441322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1141/2309] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 10:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 10:00:58.442014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1169/2310] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:00:57 2022] DELETE /identity/v3/users/3dd300436c9545faa56d60ac290bc987 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:58.452931 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b5e89-4b83-4889-9ce5-052ae31bb2c1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:58.453466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b5e89-4b83-4889-9ce5-052ae31bb2c1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:58.453962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b5e89-4b83-4889-9ce5-052ae31bb2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:58.522249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-701b5e89-4b83-4889-9ce5-052ae31bb2c1 None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:58.523578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-701b5e89-4b83-4889-9ce5-052ae31bb2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:58.561502 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:58.628137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1142/2311] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 10:00:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:58.642595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:58.644216 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:58.716970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d20c6e-ca84-4f0c-9cb2-5ef5b8fca222 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:58.722380 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a108941-1809-4ede-af2a-84eb82c38c4a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd59a351971e94aa38a22e8113207e5b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:58.723043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a108941-1809-4ede-af2a-84eb82c38c4a service glance] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:58.723328 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a108941-1809-4ede-af2a-84eb82c38c4a service glance] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:58.723606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a108941-1809-4ede-af2a-84eb82c38c4a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:58.734486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a108941-1809-4ede-af2a-84eb82c38c4a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:58.735916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a108941-1809-4ede-af2a-84eb82c38c4a service glance] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:58.747957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1170/2312] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:58.769853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b5e89-4b83-4889-9ce5-052ae31bb2c1 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:58.777937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-701b5e89-4b83-4889-9ce5-052ae31bb2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:58.901196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b5e89-4b83-4889-9ce5-052ae31bb2c1 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:58.906146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:58.907109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:58.909346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:58.909346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] PATH_INFO: `/v3/projects/4e1793ba5f584cb0a9005c50d80c5c06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:58.909346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1793ba5f584cb0a9005c50d80c5c06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:58.910907 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:59.069817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d28c5d-ec49-4373-ac7e-64d991bb3223 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.070828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d28c5d-ec49-4373-ac7e-64d991bb3223 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.071199 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d28c5d-ec49-4373-ac7e-64d991bb3223 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.073448 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1171/2313] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 10:00:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 10:00:59.079274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b12feb-1875-484d-80a0-06dc4ed5573d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.079773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b12feb-1875-484d-80a0-06dc4ed5573d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.080059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b12feb-1875-484d-80a0-06dc4ed5573d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.083408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1793ba5f584cb0a9005c50d80c5c06', 'request_id': 'req-67325b5c-3074-4770-a039-ef5b5fb8805c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:59.089109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1793ba5f584cb0a9005c50d80c5c06', 'request_id': 'req-67325b5c-3074-4770-a039-ef5b5fb8805c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:59.100278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f2b12feb-1875-484d-80a0-06dc4ed5573d None None] MFA Rules not processed for user `362797538cbd48dc9c73b09b0548959e`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:59.101516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f2b12feb-1875-484d-80a0-06dc4ed5573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:59.151138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:59.160746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1172/2314] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 10:00:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:59.179599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b12feb-1875-484d-80a0-06dc4ed5573d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:59.179599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f2b12feb-1875-484d-80a0-06dc4ed5573d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:59.187068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b12feb-1875-484d-80a0-06dc4ed5573d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:59.190858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38a3b2-70ba-4c1a-aeac-252b026e7e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:59.191506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38a3b2-70ba-4c1a-aeac-252b026e7e83 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.191815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38a3b2-70ba-4c1a-aeac-252b026e7e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.192100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38a3b2-70ba-4c1a-aeac-252b026e7e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.198940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1793ba5f584cb0a9005c50d80c5c06', 'request_id': 'req-67325b5c-3074-4770-a039-ef5b5fb8805c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:59.201796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38a3b2-70ba-4c1a-aeac-252b026e7e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:59.203072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38a3b2-70ba-4c1a-aeac-252b026e7e83 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:59.212326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1173/2315] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:00:59.214257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] The token cache is being invalidate because project 4e1793ba5f584cb0a9005c50d80c5c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:00:59.214621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:59.215322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-67325b5c-3074-4770-a039-ef5b5fb8805c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:00:59.221876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1143/2316] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:00:58 2022] DELETE /identity/v3/projects/4e1793ba5f584cb0a9005c50d80c5c06 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:00:59.222981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983be50b-3c37-4b19-bf39-2c6026a6caee None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.223283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983be50b-3c37-4b19-bf39-2c6026a6caee None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.223633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983be50b-3c37-4b19-bf39-2c6026a6caee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.245332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-983be50b-3c37-4b19-bf39-2c6026a6caee None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:59.246938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-983be50b-3c37-4b19-bf39-2c6026a6caee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:59.324385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1174/2317] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:59.332427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574014b-c0a6-46e0-a79e-bc00b1aed64d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.333019 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574014b-c0a6-46e0-a79e-bc00b1aed64d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.333518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574014b-c0a6-46e0-a79e-bc00b1aed64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.346955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fa6db-174f-4d89-b011-e8df9c7bf46f None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.347485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fa6db-174f-4d89-b011-e8df9c7bf46f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.349347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fa6db-174f-4d89-b011-e8df9c7bf46f None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.350822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1175/2318] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 10:00:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 10:00:59.356283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c574014b-c0a6-46e0-a79e-bc00b1aed64d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:59.357266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c574014b-c0a6-46e0-a79e-bc00b1aed64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:59.359178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94007-4897-4d0d-9efb-db0e7edf0a13 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.359178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94007-4897-4d0d-9efb-db0e7edf0a13 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.359703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94007-4897-4d0d-9efb-db0e7edf0a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.368990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1144/2319] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:00:59.370552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574014b-c0a6-46e0-a79e-bc00b1aed64d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:59.379817 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574014b-c0a6-46e0-a79e-bc00b1aed64d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:59.382575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-49a94007-4897-4d0d-9efb-db0e7edf0a13 None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:00:59.385321 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba6bfc-5965-43b1-b3fd-d98056c71176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:59.386355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba6bfc-5965-43b1-b3fd-d98056c71176 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.386481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-49a94007-4897-4d0d-9efb-db0e7edf0a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:59.386757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba6bfc-5965-43b1-b3fd-d98056c71176 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.387098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba6bfc-5965-43b1-b3fd-d98056c71176 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.388008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba6bfc-5965-43b1-b3fd-d98056c71176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:00:59.388403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba6bfc-5965-43b1-b3fd-d98056c71176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:59.390150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba6bfc-5965-43b1-b3fd-d98056c71176 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:59.406011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1145/2320] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:00:59.407617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba6bfc-5965-43b1-b3fd-d98056c71176 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:59.417829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba6bfc-5965-43b1-b3fd-d98056c71176 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:59.424503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3911699-a6eb-42c0-855d-63260c99b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:59.425741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3911699-a6eb-42c0-855d-63260c99b414 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.429168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3911699-a6eb-42c0-855d-63260c99b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.429168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3911699-a6eb-42c0-855d-63260c99b414 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.429168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3911699-a6eb-42c0-855d-63260c99b414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:59.430901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3911699-a6eb-42c0-855d-63260c99b414 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:59.462365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1176/2321] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 10:00:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:00:59.474087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a94007-4897-4d0d-9efb-db0e7edf0a13 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:59.475731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-49a94007-4897-4d0d-9efb-db0e7edf0a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:00:59.488974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a94007-4897-4d0d-9efb-db0e7edf0a13 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:59.498489 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d1dc1-86d3-4cdf-845e-2d1b30d9b7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:59.499877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d1dc1-86d3-4cdf-845e-2d1b30d9b7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.500252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d1dc1-86d3-4cdf-845e-2d1b30d9b7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.500623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d1dc1-86d3-4cdf-845e-2d1b30d9b7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.514404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d1dc1-86d3-4cdf-845e-2d1b30d9b7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:59.516139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d1dc1-86d3-4cdf-845e-2d1b30d9b7f2 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:59.529425 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1177/2322] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:00:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:00:59.577461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:00:59.585756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1146/2323] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:00:59 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:59.589516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d1dc1-86d3-4cdf-845e-2d1b30d9b7f2 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:59.601583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d1dc1-86d3-4cdf-845e-2d1b30d9b7f2 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:59.668851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63e07e-d453-44f7-876b-99e5b57cd99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:59.668851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63e07e-d453-44f7-876b-99e5b57cd99b admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.668851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63e07e-d453-44f7-876b-99e5b57cd99b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.669221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63e07e-d453-44f7-876b-99e5b57cd99b admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.670640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63e07e-d453-44f7-876b-99e5b57cd99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:59.673235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63e07e-d453-44f7-876b-99e5b57cd99b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:59.779603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3911699-a6eb-42c0-855d-63260c99b414 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:59.790382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1178/2324] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:00:59 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:00:59.795295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3911699-a6eb-42c0-855d-63260c99b414 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:59.802751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6d73c-5f0a-4252-a480-75e378014143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:59.803448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6d73c-5f0a-4252-a480-75e378014143 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.803736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6d73c-5f0a-4252-a480-75e378014143 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.804012 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6d73c-5f0a-4252-a480-75e378014143 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.804831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6d73c-5f0a-4252-a480-75e378014143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:59.807013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6d73c-5f0a-4252-a480-75e378014143 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:59.821243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1147/2325] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:59.824978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63e07e-d453-44f7-876b-99e5b57cd99b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:59.835801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63e07e-d453-44f7-876b-99e5b57cd99b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:59.841728 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919d974-bd1a-4447-8d0f-8e264d01ac4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:59.842432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919d974-bd1a-4447-8d0f-8e264d01ac4b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.842723 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919d974-bd1a-4447-8d0f-8e264d01ac4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.843021 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919d974-bd1a-4447-8d0f-8e264d01ac4b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.843719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919d974-bd1a-4447-8d0f-8e264d01ac4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:59.846379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919d974-bd1a-4447-8d0f-8e264d01ac4b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:00:59.857500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1179/2326] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:00:59.862167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6d73c-5f0a-4252-a480-75e378014143 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:00:59.873300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6d73c-5f0a-4252-a480-75e378014143 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:00:59.881966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104b22b-f672-48bc-b25c-59ff2579b292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:00:59.883190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104b22b-f672-48bc-b25c-59ff2579b292 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:00:59.883700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104b22b-f672-48bc-b25c-59ff2579b292 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:00:59.884312 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104b22b-f672-48bc-b25c-59ff2579b292 admin admin] PATH_INFO: `/v3/projects/d60aa20cdf624925871544346abb9baf/users/96e4edd8c193474cba90fc5cde6813ec/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:00:59.902454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104b22b-f672-48bc-b25c-59ff2579b292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60aa20cdf624925871544346abb9baf, user_id=96e4edd8c193474cba90fc5cde6813ec, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:00:59.905216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104b22b-f672-48bc-b25c-59ff2579b292 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:00.014527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1148/2327] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:00:59 2022] PUT /identity/v3/projects/d60aa20cdf624925871544346abb9baf/users/96e4edd8c193474cba90fc5cde6813ec/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:00.024491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5e6cd-73a8-45a8-9ab0-d00c52dc7d81 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:00.024787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5e6cd-73a8-45a8-9ab0-d00c52dc7d81 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:00.025065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5e6cd-73a8-45a8-9ab0-d00c52dc7d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:00.054022 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-69c5e6cd-73a8-45a8-9ab0-d00c52dc7d81 None None] MFA Rules not processed for user `96e4edd8c193474cba90fc5cde6813ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:00.055248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-69c5e6cd-73a8-45a8-9ab0-d00c52dc7d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:00.123010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1180/2328] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:00.134508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104b22b-f672-48bc-b25c-59ff2579b292 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:00.136468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b104b22b-f672-48bc-b25c-59ff2579b292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:00.235711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104b22b-f672-48bc-b25c-59ff2579b292 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:00.242742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914fe72-2fb3-4795-bbdc-8520affb66e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:00.243766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914fe72-2fb3-4795-bbdc-8520affb66e4 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:00.245004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914fe72-2fb3-4795-bbdc-8520affb66e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:00.246561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914fe72-2fb3-4795-bbdc-8520affb66e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:00.262590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914fe72-2fb3-4795-bbdc-8520affb66e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:00.264558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914fe72-2fb3-4795-bbdc-8520affb66e4 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:00.276571 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1149/2329] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:01:00.304587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5e6cd-73a8-45a8-9ab0-d00c52dc7d81 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:00.368782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5e6cd-73a8-45a8-9ab0-d00c52dc7d81 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:00.375544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:00.376530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:00.376961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:00.377246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] PATH_INFO: `/v3/users/96e4edd8c193474cba90fc5cde6813ec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:00.382365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e4edd8c193474cba90fc5cde6813ec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:00.384753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:00.688831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e4edd8c193474cba90fc5cde6813ec'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:00.689516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e4edd8c193474cba90fc5cde6813ec'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:00.697804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e4edd8c193474cba90fc5cde6813ec'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:00.704970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:00.719970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e4edd8c193474cba90fc5cde6813ec'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:00.799148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e4edd8c193474cba90fc5cde6813ec'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:00.807411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1181/2330] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:01:00 2022] DELETE /identity/v3/users/96e4edd8c193474cba90fc5cde6813ec => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:00.812049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cf325-de62-4500-8d75-7050f10d3060 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:00.812565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cf325-de62-4500-8d75-7050f10d3060 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:00.812741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cf325-de62-4500-8d75-7050f10d3060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:00.831818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b63cf325-de62-4500-8d75-7050f10d3060 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:00.833130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b63cf325-de62-4500-8d75-7050f10d3060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:00.899169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1150/2331] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:01:00.910531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:00.911991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:00.973187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff581ac-9f3c-4159-8ed7-ce607dc51c9f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:00.976802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7347f2-4522-44d8-bbf0-b12ca51646a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:00.977449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7347f2-4522-44d8-bbf0-b12ca51646a7 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:00.977735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7347f2-4522-44d8-bbf0-b12ca51646a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:00.978047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7347f2-4522-44d8-bbf0-b12ca51646a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:00.988109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7347f2-4522-44d8-bbf0-b12ca51646a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:00.989230 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7347f2-4522-44d8-bbf0-b12ca51646a7 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:00.999377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1182/2332] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:01:01.557601 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:01.696715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cf325-de62-4500-8d75-7050f10d3060 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:01.698501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b63cf325-de62-4500-8d75-7050f10d3060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:01.745108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cf325-de62-4500-8d75-7050f10d3060 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:01.751890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:01.752779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:01.770212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:01.770212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] PATH_INFO: `/v3/projects/d60aa20cdf624925871544346abb9baf` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:01.770212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d60aa20cdf624925871544346abb9baf)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:01.770212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:01.906214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd60aa20cdf624925871544346abb9baf', 'request_id': 'req-098838d4-0b64-40fb-96ac-83d7f176d86f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:01.912455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd60aa20cdf624925871544346abb9baf', 'request_id': 'req-098838d4-0b64-40fb-96ac-83d7f176d86f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:01.917350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:02.008851 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd60aa20cdf624925871544346abb9baf', 'request_id': 'req-098838d4-0b64-40fb-96ac-83d7f176d86f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:02.021334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] The token cache is being invalidate because project d60aa20cdf624925871544346abb9baf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:01:02.021947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:02.022460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-098838d4-0b64-40fb-96ac-83d7f176d86f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:02.024120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1151/2333] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:01:01 2022] DELETE /identity/v3/projects/d60aa20cdf624925871544346abb9baf => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:02.030727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab9d19-7c0f-4e01-80c7-e652149b0311 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.031368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab9d19-7c0f-4e01-80c7-e652149b0311 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.031877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab9d19-7c0f-4e01-80c7-e652149b0311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:02.050984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1cab9d19-7c0f-4e01-80c7-e652149b0311 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:02.052228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1cab9d19-7c0f-4e01-80c7-e652149b0311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:02.112143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1183/2334] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:02.120144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b08b6-90a0-4c1f-8424-8ede20e3a7cc None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.120437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b08b6-90a0-4c1f-8424-8ede20e3a7cc None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.120749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b08b6-90a0-4c1f-8424-8ede20e3a7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:02.140479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a68b08b6-90a0-4c1f-8424-8ede20e3a7cc None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:02.141684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a68b08b6-90a0-4c1f-8424-8ede20e3a7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:02.153678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1152/2335] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:02.154838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab9d19-7c0f-4e01-80c7-e652149b0311 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:02.167267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab9d19-7c0f-4e01-80c7-e652149b0311 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:02.174893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952949e-1998-45fb-8b79-d90d06e5d93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:02.175639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952949e-1998-45fb-8b79-d90d06e5d93b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.176055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952949e-1998-45fb-8b79-d90d06e5d93b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.176335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952949e-1998-45fb-8b79-d90d06e5d93b admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:02.177070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952949e-1998-45fb-8b79-d90d06e5d93b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:01:02.177379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952949e-1998-45fb-8b79-d90d06e5d93b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:02.179095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952949e-1998-45fb-8b79-d90d06e5d93b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:02.190944 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1184/2336] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:01:02.195425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b08b6-90a0-4c1f-8424-8ede20e3a7cc None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:02.206984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b08b6-90a0-4c1f-8424-8ede20e3a7cc None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:02.211700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c166f92-4dba-4cab-8bd9-948b525d8a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:02.212381 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c166f92-4dba-4cab-8bd9-948b525d8a48 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.212694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c166f92-4dba-4cab-8bd9-948b525d8a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.213488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c166f92-4dba-4cab-8bd9-948b525d8a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:02.214226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c166f92-4dba-4cab-8bd9-948b525d8a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:02.216189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c166f92-4dba-4cab-8bd9-948b525d8a48 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:02.336836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1153/2337] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:01:02 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:01:02.338893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952949e-1998-45fb-8b79-d90d06e5d93b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:02.349502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952949e-1998-45fb-8b79-d90d06e5d93b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:02.408781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ea3c1-67c7-4cea-b66e-f57ca8cc1abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:02.409389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ea3c1-67c7-4cea-b66e-f57ca8cc1abb admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.409676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ea3c1-67c7-4cea-b66e-f57ca8cc1abb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.409995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ea3c1-67c7-4cea-b66e-f57ca8cc1abb admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:02.410693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ea3c1-67c7-4cea-b66e-f57ca8cc1abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:02.412640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ea3c1-67c7-4cea-b66e-f57ca8cc1abb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:02.565668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1185/2338] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:01:02 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:01:02.574246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c166f92-4dba-4cab-8bd9-948b525d8a48 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:02.585764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c166f92-4dba-4cab-8bd9-948b525d8a48 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:02.590553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c7545-00bd-4a6e-ad20-9f97affb6de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:02.591210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c7545-00bd-4a6e-ad20-9f97affb6de0 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.591492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c7545-00bd-4a6e-ad20-9f97affb6de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.591763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c7545-00bd-4a6e-ad20-9f97affb6de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:02.592539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c7545-00bd-4a6e-ad20-9f97affb6de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:02.594164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c7545-00bd-4a6e-ad20-9f97affb6de0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:02.604428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1154/2339] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:02.608644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ea3c1-67c7-4cea-b66e-f57ca8cc1abb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:02.618708 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ea3c1-67c7-4cea-b66e-f57ca8cc1abb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:02.623801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc47ab-4e86-44c4-a167-56b65094029a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:02.624814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc47ab-4e86-44c4-a167-56b65094029a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.624814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc47ab-4e86-44c4-a167-56b65094029a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.625019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc47ab-4e86-44c4-a167-56b65094029a admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:02.625720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc47ab-4e86-44c4-a167-56b65094029a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:02.627348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc47ab-4e86-44c4-a167-56b65094029a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:02.632972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:02.640097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1186/2340] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:02.644744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c7545-00bd-4a6e-ad20-9f97affb6de0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:02.655835 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c7545-00bd-4a6e-ad20-9f97affb6de0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:02.660626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1223b-ac2e-4039-9be5-71bac87b2170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:02.661515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1223b-ac2e-4039-9be5-71bac87b2170 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.661826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1223b-ac2e-4039-9be5-71bac87b2170 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.665514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af1223b-ac2e-4039-9be5-71bac87b2170 admin admin] PATH_INFO: `/v3/projects/5476f21b29944eb69e32a2d97b328aed/users/646fecf4405c41a0bc062c6b54fd311c/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:02.677200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1223b-ac2e-4039-9be5-71bac87b2170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5476f21b29944eb69e32a2d97b328aed, user_id=646fecf4405c41a0bc062c6b54fd311c, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:02.679901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af1223b-ac2e-4039-9be5-71bac87b2170 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:02.787539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1155/2341] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:01:02 2022] PUT /identity/v3/projects/5476f21b29944eb69e32a2d97b328aed/users/646fecf4405c41a0bc062c6b54fd311c/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:02.789483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73167-c886-494d-8136-349e24f24609 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.789871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73167-c886-494d-8136-349e24f24609 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.790066 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc73167-c886-494d-8136-349e24f24609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:02.821471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0fc73167-c886-494d-8136-349e24f24609 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:02.823579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0fc73167-c886-494d-8136-349e24f24609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:02.891797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1187/2342] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:02.901628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1223b-ac2e-4039-9be5-71bac87b2170 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:02.903498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7af1223b-ac2e-4039-9be5-71bac87b2170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:02.969997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af1223b-ac2e-4039-9be5-71bac87b2170 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:02.982848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8f264-f7d9-4a68-9272-f756377d712e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:02.983140 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f264-f7d9-4a68-9272-f756377d712e service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:02.983433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f264-f7d9-4a68-9272-f756377d712e service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:02.983714 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8f264-f7d9-4a68-9272-f756377d712e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:03.000187 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f264-f7d9-4a68-9272-f756377d712e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:03.001566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf8f264-f7d9-4a68-9272-f756377d712e service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:03.014388 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1156/2343] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:01:03.837404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73167-c886-494d-8136-349e24f24609 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:03.840294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0fc73167-c886-494d-8136-349e24f24609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:03.855924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc73167-c886-494d-8136-349e24f24609 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:03.861606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:03.862894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:03.863187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:03.863515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] PATH_INFO: `/v3/users/b37fcf4ca8984d0cb3582410324c4901` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:03.864906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37fcf4ca8984d0cb3582410324c4901)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:03.867067 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:04.050958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37fcf4ca8984d0cb3582410324c4901'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:04.051678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37fcf4ca8984d0cb3582410324c4901'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:04.058810 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37fcf4ca8984d0cb3582410324c4901'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:04.074350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37fcf4ca8984d0cb3582410324c4901'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:04.172174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37fcf4ca8984d0cb3582410324c4901'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:04.178818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1188/2344] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:01:03 2022] DELETE /identity/v3/users/b37fcf4ca8984d0cb3582410324c4901 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:04.183384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c41c11-1bb3-475d-9416-e63230f1c398 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:04.183869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c41c11-1bb3-475d-9416-e63230f1c398 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:04.183978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c41c11-1bb3-475d-9416-e63230f1c398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:04.201083 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-b1c41c11-1bb3-475d-9416-e63230f1c398 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:04.202322 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b1c41c11-1bb3-475d-9416-e63230f1c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:04.298143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1157/2345] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:04.308028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:04.309587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:04.375490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0755616-a1f8-4e5c-8ada-3662524466ff admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:04.379343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832552e-a914-427f-b803-2b3b23b58052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:04.380074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832552e-a914-427f-b803-2b3b23b58052 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:04.380414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832552e-a914-427f-b803-2b3b23b58052 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:04.380699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832552e-a914-427f-b803-2b3b23b58052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:04.392298 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832552e-a914-427f-b803-2b3b23b58052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:04.393714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832552e-a914-427f-b803-2b3b23b58052 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:04.405707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1189/2346] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:01:05.125075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c41c11-1bb3-475d-9416-e63230f1c398 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:05.127568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b1c41c11-1bb3-475d-9416-e63230f1c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:05.149014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c41c11-1bb3-475d-9416-e63230f1c398 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:05.158218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:05.160526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:05.161226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:05.161925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] PATH_INFO: `/v3/projects/31b7ad0a70d34261b6b8aafcdbc2c5c7` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:05.165015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b7ad0a70d34261b6b8aafcdbc2c5c7)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:05.167712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:05.326865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:05.348954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b7ad0a70d34261b6b8aafcdbc2c5c7', 'request_id': 'req-184b06d5-aa02-447e-894f-b17ae5e0c4bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:05.356768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b7ad0a70d34261b6b8aafcdbc2c5c7', 'request_id': 'req-184b06d5-aa02-447e-894f-b17ae5e0c4bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:05.530620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b7ad0a70d34261b6b8aafcdbc2c5c7', 'request_id': 'req-184b06d5-aa02-447e-894f-b17ae5e0c4bb'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:05.542998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] The token cache is being invalidate because project 31b7ad0a70d34261b6b8aafcdbc2c5c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:01:05.543296 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:05.543936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-184b06d5-aa02-447e-894f-b17ae5e0c4bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:05.545483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1158/2347] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:01:05 2022] DELETE /identity/v3/projects/31b7ad0a70d34261b6b8aafcdbc2c5c7 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:05.550620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb922a2-53ec-4d4b-8473-73f1175bf3f3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:05.551482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb922a2-53ec-4d4b-8473-73f1175bf3f3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:05.551827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb922a2-53ec-4d4b-8473-73f1175bf3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:05.571642 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9cb922a2-53ec-4d4b-8473-73f1175bf3f3 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:05.573162 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9cb922a2-53ec-4d4b-8473-73f1175bf3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:05.583258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:05.646009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1190/2348] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:05.655143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be6a6e-4f21-4c62-b28a-1425e31094be None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:05.655436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be6a6e-4f21-4c62-b28a-1425e31094be None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:05.655754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be6a6e-4f21-4c62-b28a-1425e31094be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:05.674841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-50be6a6e-4f21-4c62-b28a-1425e31094be None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:05.676347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-50be6a6e-4f21-4c62-b28a-1425e31094be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:05.688651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1159/2349] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:05.690979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb922a2-53ec-4d4b-8473-73f1175bf3f3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:05.704257 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb922a2-53ec-4d4b-8473-73f1175bf3f3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:05.708935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb30b75-7890-40a3-a6b5-4b9d9cbfe9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:05.709586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb30b75-7890-40a3-a6b5-4b9d9cbfe9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:05.709868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb30b75-7890-40a3-a6b5-4b9d9cbfe9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:05.710175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb30b75-7890-40a3-a6b5-4b9d9cbfe9a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:05.710906 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb30b75-7890-40a3-a6b5-4b9d9cbfe9a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:01:05.711211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb30b75-7890-40a3-a6b5-4b9d9cbfe9a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:05.714021 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb30b75-7890-40a3-a6b5-4b9d9cbfe9a3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:05.733662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1191/2350] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:01:05.737765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be6a6e-4f21-4c62-b28a-1425e31094be None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:05.748864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be6a6e-4f21-4c62-b28a-1425e31094be None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:05.755212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e34d6-94af-41eb-91ab-dbbf6b689ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:05.763323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e34d6-94af-41eb-91ab-dbbf6b689ebd admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:05.763323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e34d6-94af-41eb-91ab-dbbf6b689ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:05.763323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e34d6-94af-41eb-91ab-dbbf6b689ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:05.763323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e34d6-94af-41eb-91ab-dbbf6b689ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:05.763323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e34d6-94af-41eb-91ab-dbbf6b689ebd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:05.897918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1160/2351] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:01:05 2022] POST /identity/v3/projects => generated 397 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:01:05.900825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb30b75-7890-40a3-a6b5-4b9d9cbfe9a3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:05.912619 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb30b75-7890-40a3-a6b5-4b9d9cbfe9a3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:06.054747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5199a95-ecb9-49a3-bfdd-a456897931b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:06.055621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5199a95-ecb9-49a3-bfdd-a456897931b4 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:06.055738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5199a95-ecb9-49a3-bfdd-a456897931b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:06.055989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5199a95-ecb9-49a3-bfdd-a456897931b4 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:06.057166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5199a95-ecb9-49a3-bfdd-a456897931b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:06.061416 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5199a95-ecb9-49a3-bfdd-a456897931b4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:06.162578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1192/2352] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:01:05 2022] POST /identity/v3/users => generated 299 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:01:06.166747 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e34d6-94af-41eb-91ab-dbbf6b689ebd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:06.181134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e34d6-94af-41eb-91ab-dbbf6b689ebd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:06.200151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4e4b9-09e7-4c42-bf21-c03bf7e8a8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:06.200850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4e4b9-09e7-4c42-bf21-c03bf7e8a8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:06.201342 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4e4b9-09e7-4c42-bf21-c03bf7e8a8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:06.201633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4e4b9-09e7-4c42-bf21-c03bf7e8a8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:06.202458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4e4b9-09e7-4c42-bf21-c03bf7e8a8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:06.204099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4e4b9-09e7-4c42-bf21-c03bf7e8a8a6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:06.233383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1161/2353] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:06.234354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5199a95-ecb9-49a3-bfdd-a456897931b4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:06.245786 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5199a95-ecb9-49a3-bfdd-a456897931b4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:06.251769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af2f86-70d6-48a6-be2c-ac1f95880b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:06.252561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af2f86-70d6-48a6-be2c-ac1f95880b3b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:06.252881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af2f86-70d6-48a6-be2c-ac1f95880b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:06.253365 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af2f86-70d6-48a6-be2c-ac1f95880b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:06.254348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af2f86-70d6-48a6-be2c-ac1f95880b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:06.256382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af2f86-70d6-48a6-be2c-ac1f95880b3b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:06.269119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1193/2354] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:06.272892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4e4b9-09e7-4c42-bf21-c03bf7e8a8a6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:06.288191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4e4b9-09e7-4c42-bf21-c03bf7e8a8a6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:06.297159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e55709-89e1-48f1-bb49-34f5b776eee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:06.298429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e55709-89e1-48f1-bb49-34f5b776eee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:06.299029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e55709-89e1-48f1-bb49-34f5b776eee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:06.299555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e55709-89e1-48f1-bb49-34f5b776eee3 admin admin] PATH_INFO: `/v3/projects/dcba6bfc4cb84b24a5c7cae93bf80009/users/d095608f007e458d9e9c6db9f9e7990d/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:06.316606 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e55709-89e1-48f1-bb49-34f5b776eee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcba6bfc4cb84b24a5c7cae93bf80009, user_id=d095608f007e458d9e9c6db9f9e7990d, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:06.319940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e55709-89e1-48f1-bb49-34f5b776eee3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:06.426009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1162/2355] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:01:06 2022] PUT /identity/v3/projects/dcba6bfc4cb84b24a5c7cae93bf80009/users/d095608f007e458d9e9c6db9f9e7990d/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:06.442097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6e1ec-9baa-4129-9f9b-d2b4e8ac424f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:06.442392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6e1ec-9baa-4129-9f9b-d2b4e8ac424f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:06.442681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6e1ec-9baa-4129-9f9b-d2b4e8ac424f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:06.472560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-61f6e1ec-9baa-4129-9f9b-d2b4e8ac424f None None] MFA Rules not processed for user `d095608f007e458d9e9c6db9f9e7990d`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:06.473838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-61f6e1ec-9baa-4129-9f9b-d2b4e8ac424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:06.560085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1194/2356] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:06.571532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e55709-89e1-48f1-bb49-34f5b776eee3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:06.574127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-75e55709-89e1-48f1-bb49-34f5b776eee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:06.633197 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e55709-89e1-48f1-bb49-34f5b776eee3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:06.637749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc3286-03eb-44a7-a02e-15d622ea811e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:06.638445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc3286-03eb-44a7-a02e-15d622ea811e service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:06.638774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc3286-03eb-44a7-a02e-15d622ea811e service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:06.639425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc3286-03eb-44a7-a02e-15d622ea811e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:06.648930 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc3286-03eb-44a7-a02e-15d622ea811e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:06.650551 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc3286-03eb-44a7-a02e-15d622ea811e service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:06.660538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1163/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:01:06.900777 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64135d-f7c4-4e3f-923e-d9596ed5830f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:06.901138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64135d-f7c4-4e3f-923e-d9596ed5830f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:06.903376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64135d-f7c4-4e3f-923e-d9596ed5830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:06.923312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:06.968634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1b64135d-f7c4-4e3f-923e-d9596ed5830f None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:06.970161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1b64135d-f7c4-4e3f-923e-d9596ed5830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:07.046217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1195/2358] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 10:01:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:07.075222 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc3286-03eb-44a7-a02e-15d622ea811e service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:07.076784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4cbc3286-03eb-44a7-a02e-15d622ea811e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:07.157765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:07.233328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc3286-03eb-44a7-a02e-15d622ea811e service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:07.256353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fab2a3-5ce0-4ede-a4ce-5a031f8dfa5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:07.260555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fab2a3-5ce0-4ede-a4ce-5a031f8dfa5e service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:07.260555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fab2a3-5ce0-4ede-a4ce-5a031f8dfa5e service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:07.260555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fab2a3-5ce0-4ede-a4ce-5a031f8dfa5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:07.281170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fab2a3-5ce0-4ede-a4ce-5a031f8dfa5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:07.284348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fab2a3-5ce0-4ede-a4ce-5a031f8dfa5e service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:07.315905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1164/2359] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:01:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 03 10:01:10.680389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbc1b3-e997-4054-84a0-607bf834402d None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:10.680953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbc1b3-e997-4054-84a0-607bf834402d None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:10.680953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbc1b3-e997-4054-84a0-607bf834402d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:10.712832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b1dbc1b3-e997-4054-84a0-607bf834402d None None] MFA Rules not processed for user `646fecf4405c41a0bc062c6b54fd311c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:10.714077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b1dbc1b3-e997-4054-84a0-607bf834402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:10.788003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1196/2360] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:10.793136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fab2a3-5ce0-4ede-a4ce-5a031f8dfa5e service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:10.794659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-96fab2a3-5ce0-4ede-a4ce-5a031f8dfa5e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:10.860018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fab2a3-5ce0-4ede-a4ce-5a031f8dfa5e service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:10.864588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a3f80-deac-42b2-8847-cafc45090fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:10.865244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a3f80-deac-42b2-8847-cafc45090fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:10.865534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a3f80-deac-42b2-8847-cafc45090fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:10.865807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a3f80-deac-42b2-8847-cafc45090fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:10.866535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a3f80-deac-42b2-8847-cafc45090fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:10.868531 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a3f80-deac-42b2-8847-cafc45090fe6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:10.956792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1165/2361] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:01:10 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:01:10.960945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbc1b3-e997-4054-84a0-607bf834402d None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:10.972514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbc1b3-e997-4054-84a0-607bf834402d None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:11.030927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6a370-d82e-470b-8867-e10f82ce5f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:11.031713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6a370-d82e-470b-8867-e10f82ce5f88 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:11.032001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6a370-d82e-470b-8867-e10f82ce5f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:11.032286 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6a370-d82e-470b-8867-e10f82ce5f88 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:11.032990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6a370-d82e-470b-8867-e10f82ce5f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:11.035156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6a370-d82e-470b-8867-e10f82ce5f88 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:11.116229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1197/2362] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:01:10 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:01:11.117676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a3f80-deac-42b2-8847-cafc45090fe6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:11.128582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a3f80-deac-42b2-8847-cafc45090fe6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:11.133588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c736c-235b-469c-9a61-96d040ada107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:11.134792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c736c-235b-469c-9a61-96d040ada107 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:11.140352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c736c-235b-469c-9a61-96d040ada107 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:11.140352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c736c-235b-469c-9a61-96d040ada107 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:11.140352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c736c-235b-469c-9a61-96d040ada107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:11.140352 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c736c-235b-469c-9a61-96d040ada107 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:11.149782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1166/2363] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:11.154502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6a370-d82e-470b-8867-e10f82ce5f88 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:11.159674 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:11.164756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6a370-d82e-470b-8867-e10f82ce5f88 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:11.170718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d00146-fa9c-4327-b16b-222886812a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:11.171635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d00146-fa9c-4327-b16b-222886812a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:11.171912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d00146-fa9c-4327-b16b-222886812a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:11.172184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d00146-fa9c-4327-b16b-222886812a79 admin admin] PATH_INFO: `/v3/projects/587f4a774d5e46d1a61cbe83c1d42631/users/6b55eed0af3e4b838d62750fee9cbe98/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:11.189792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d00146-fa9c-4327-b16b-222886812a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587f4a774d5e46d1a61cbe83c1d42631, user_id=6b55eed0af3e4b838d62750fee9cbe98, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:11.192570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d00146-fa9c-4327-b16b-222886812a79 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:11.297096 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1198/2364] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:01:11 2022] PUT /identity/v3/projects/587f4a774d5e46d1a61cbe83c1d42631/users/6b55eed0af3e4b838d62750fee9cbe98/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:11.301132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c736c-235b-469c-9a61-96d040ada107 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:11.311287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c736c-235b-469c-9a61-96d040ada107 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:11.374937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee017e9-2275-43f0-8fd9-ca3bd9f50b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:11.375645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee017e9-2275-43f0-8fd9-ca3bd9f50b9e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:11.376097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee017e9-2275-43f0-8fd9-ca3bd9f50b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:11.376380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee017e9-2275-43f0-8fd9-ca3bd9f50b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:11.377112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee017e9-2275-43f0-8fd9-ca3bd9f50b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:11.378759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee017e9-2275-43f0-8fd9-ca3bd9f50b9e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:11.390433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1167/2365] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:11.394946 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d00146-fa9c-4327-b16b-222886812a79 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:11.409665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d00146-fa9c-4327-b16b-222886812a79 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:11.418951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5846f4-64d8-497d-ab82-df60ba5487d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:11.420018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5846f4-64d8-497d-ab82-df60ba5487d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:11.420479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5846f4-64d8-497d-ab82-df60ba5487d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:11.420846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5846f4-64d8-497d-ab82-df60ba5487d6 admin admin] PATH_INFO: `/v3/projects/587f4a774d5e46d1a61cbe83c1d42631/users/6b55eed0af3e4b838d62750fee9cbe98/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:11.425819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5846f4-64d8-497d-ab82-df60ba5487d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587f4a774d5e46d1a61cbe83c1d42631, user_id=6b55eed0af3e4b838d62750fee9cbe98, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:11.425819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5846f4-64d8-497d-ab82-df60ba5487d6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:11.436921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1199/2366] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:01:11 2022] PUT /identity/v3/projects/587f4a774d5e46d1a61cbe83c1d42631/users/6b55eed0af3e4b838d62750fee9cbe98/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:16.590328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee017e9-2275-43f0-8fd9-ca3bd9f50b9e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:16.604756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee017e9-2275-43f0-8fd9-ca3bd9f50b9e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:16.690214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:16.754649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:16.755548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:16.755830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:16.756105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] PATH_INFO: `/v3/users/d095608f007e458d9e9c6db9f9e7990d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:16.763627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d095608f007e458d9e9c6db9f9e7990d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:16.765914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:17.015374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd095608f007e458d9e9c6db9f9e7990d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:17.018465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd095608f007e458d9e9c6db9f9e7990d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:17.075349 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd095608f007e458d9e9c6db9f9e7990d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:17.096324 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd095608f007e458d9e9c6db9f9e7990d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:17.183209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd095608f007e458d9e9c6db9f9e7990d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:17.193579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1168/2367] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:01:16 2022] DELETE /identity/v3/users/d095608f007e458d9e9c6db9f9e7990d => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:17.198567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0e9b5-4608-4fb3-bf6a-4fbcf7363c23 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:17.200373 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0e9b5-4608-4fb3-bf6a-4fbcf7363c23 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:17.200857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0e9b5-4608-4fb3-bf6a-4fbcf7363c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:17.242858 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2af0e9b5-4608-4fb3-bf6a-4fbcf7363c23 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:17.245789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2af0e9b5-4608-4fb3-bf6a-4fbcf7363c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:17.487044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1200/2368] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:17.493341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:17.496476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:17.634820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1338ff-17b5-43b5-96a0-6c49c1705ef5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:17.640386 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffabf0-7be5-450c-bddc-5b27e1e0618f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:17.641269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffabf0-7be5-450c-bddc-5b27e1e0618f service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:17.641730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffabf0-7be5-450c-bddc-5b27e1e0618f service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:17.642200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffabf0-7be5-450c-bddc-5b27e1e0618f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:17.654971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffabf0-7be5-450c-bddc-5b27e1e0618f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:17.656742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffabf0-7be5-450c-bddc-5b27e1e0618f service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:17.683130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1169/2369] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:01:18.870065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0e9b5-4608-4fb3-bf6a-4fbcf7363c23 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:18.873161 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2af0e9b5-4608-4fb3-bf6a-4fbcf7363c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:18.894871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0e9b5-4608-4fb3-bf6a-4fbcf7363c23 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:18.910740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:18.915885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:18.916207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:18.916496 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] PATH_INFO: `/v3/projects/dcba6bfc4cb84b24a5c7cae93bf80009` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:18.934300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcba6bfc4cb84b24a5c7cae93bf80009)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:18.936620 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:19.208933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcba6bfc4cb84b24a5c7cae93bf80009', 'request_id': 'req-556c9bdd-94c3-4df3-909c-cd6201c841a3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:19.216197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcba6bfc4cb84b24a5c7cae93bf80009', 'request_id': 'req-556c9bdd-94c3-4df3-909c-cd6201c841a3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:19.304409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcba6bfc4cb84b24a5c7cae93bf80009', 'request_id': 'req-556c9bdd-94c3-4df3-909c-cd6201c841a3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:19.344853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] The token cache is being invalidate because project dcba6bfc4cb84b24a5c7cae93bf80009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:01:19.345570 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:19.346622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:19.365944 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1201/2370] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:01:18 2022] DELETE /identity/v3/projects/dcba6bfc4cb84b24a5c7cae93bf80009 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:20.191789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb034ca5-8a26-4166-a7f8-e75b514f71bb None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:20.193009 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb034ca5-8a26-4166-a7f8-e75b514f71bb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:20.193329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb034ca5-8a26-4166-a7f8-e75b514f71bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:20.222192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-eb034ca5-8a26-4166-a7f8-e75b514f71bb None None] MFA Rules not processed for user `6b55eed0af3e4b838d62750fee9cbe98`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:20.223431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-eb034ca5-8a26-4166-a7f8-e75b514f71bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:20.278061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:20.308207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1170/2371] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:01:20.320326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:20.321788 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:20.375555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:20.401632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c9bdd-94c3-4df3-909c-cd6201c841a3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:20.407498 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbe0b1-fab5-484c-a90c-48f4810f809c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:20.408152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbe0b1-fab5-484c-a90c-48f4810f809c service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:20.408532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbe0b1-fab5-484c-a90c-48f4810f809c service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:20.408811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbe0b1-fab5-484c-a90c-48f4810f809c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:20.410714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8cbbe0b1-fab5-484c-a90c-48f4810f809c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:20.470853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbe0b1-fab5-484c-a90c-48f4810f809c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:20.472429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbe0b1-fab5-484c-a90c-48f4810f809c service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:20.481813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1202/2372] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:01:26.299062 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb034ca5-8a26-4166-a7f8-e75b514f71bb None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:26.300548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-eb034ca5-8a26-4166-a7f8-e75b514f71bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:26.406781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb034ca5-8a26-4166-a7f8-e75b514f71bb None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:26.411849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:26.412576 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:26.412981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:26.413270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:26.416586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:26.429097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:26.437636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:26.445083 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1171/2373] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:01:29.714472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5733373-cc66-4800-ae5b-3d0003c47913 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:29.715034 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5733373-cc66-4800-ae5b-3d0003c47913 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:29.715034 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5733373-cc66-4800-ae5b-3d0003c47913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:29.744805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c5733373-cc66-4800-ae5b-3d0003c47913 None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:29.746158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c5733373-cc66-4800-ae5b-3d0003c47913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:29.803576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1203/2374] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 10:01:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:29.829478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:29.829478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:29.897985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2293dc3-a892-497b-9501-c103f8e251c8 service placement] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:29.903860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47849335-5ca7-40cd-b4bc-6f43133ba1a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:29.904453 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47849335-5ca7-40cd-b4bc-6f43133ba1a5 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:29.904739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47849335-5ca7-40cd-b4bc-6f43133ba1a5 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:29.905022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47849335-5ca7-40cd-b4bc-6f43133ba1a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:29.913473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47849335-5ca7-40cd-b4bc-6f43133ba1a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:29.915168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47849335-5ca7-40cd-b4bc-6f43133ba1a5 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:29.923834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1172/2375] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:01:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:01:30.887908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef01743-03d8-4369-9ce6-390f854fe50e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:30.888388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef01743-03d8-4369-9ce6-390f854fe50e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:30.888488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef01743-03d8-4369-9ce6-390f854fe50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:30.906763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4ef01743-03d8-4369-9ce6-390f854fe50e None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:30.907975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4ef01743-03d8-4369-9ce6-390f854fe50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:30.915767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1204/2376] 10.222.0.42 () {58 vars in 967 bytes} [Fri Jun 3 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:30.928153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47849335-5ca7-40cd-b4bc-6f43133ba1a5 service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:30.938550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47849335-5ca7-40cd-b4bc-6f43133ba1a5 service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:30.943535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66aa86-e2e0-4629-8282-83ee6f586ac0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:30.944160 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66aa86-e2e0-4629-8282-83ee6f586ac0 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:30.944441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66aa86-e2e0-4629-8282-83ee6f586ac0 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:30.944720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66aa86-e2e0-4629-8282-83ee6f586ac0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:30.952794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66aa86-e2e0-4629-8282-83ee6f586ac0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:30.954516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66aa86-e2e0-4629-8282-83ee6f586ac0 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:30.964211 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1173/2377] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:01:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:01:31.743683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b642ec-6632-47ff-b4e4-82c56d6f0b2f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:31.743973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b642ec-6632-47ff-b4e4-82c56d6f0b2f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:31.744251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b642ec-6632-47ff-b4e4-82c56d6f0b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:31.765369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-41b642ec-6632-47ff-b4e4-82c56d6f0b2f None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:31.766945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-41b642ec-6632-47ff-b4e4-82c56d6f0b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:31.775098 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1205/2378] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 10:01:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:31.788708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66aa86-e2e0-4629-8282-83ee6f586ac0 service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:31.803792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66aa86-e2e0-4629-8282-83ee6f586ac0 service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:31.812092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef227c-3f5a-4d6f-8483-af2fc0ff3e32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:31.812832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef227c-3f5a-4d6f-8483-af2fc0ff3e32 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:31.813224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef227c-3f5a-4d6f-8483-af2fc0ff3e32 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:31.813603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef227c-3f5a-4d6f-8483-af2fc0ff3e32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:31.824484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef227c-3f5a-4d6f-8483-af2fc0ff3e32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:31.826841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef227c-3f5a-4d6f-8483-af2fc0ff3e32 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:31.835197 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1174/2379] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:01:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:01:32.570279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab18a4c-b194-469c-a70a-9c44dbfe04fd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:32.570722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab18a4c-b194-469c-a70a-9c44dbfe04fd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:32.570827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab18a4c-b194-469c-a70a-9c44dbfe04fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:32.589697 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-dab18a4c-b194-469c-a70a-9c44dbfe04fd None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:32.590970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dab18a4c-b194-469c-a70a-9c44dbfe04fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:32.598389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1206/2380] 10.222.0.42 () {58 vars in 967 bytes} [Fri Jun 3 10:01:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:32.610353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef227c-3f5a-4d6f-8483-af2fc0ff3e32 service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:32.620820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef227c-3f5a-4d6f-8483-af2fc0ff3e32 service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:32.625844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfce14-5c95-4610-8776-535162a077d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:32.626516 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfce14-5c95-4610-8776-535162a077d2 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:32.626806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfce14-5c95-4610-8776-535162a077d2 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:32.627084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfce14-5c95-4610-8776-535162a077d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:32.635928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfce14-5c95-4610-8776-535162a077d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:32.637511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfce14-5c95-4610-8776-535162a077d2 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:32.642050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:32.646063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1175/2381] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:01:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:01:35.455807 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab18a4c-b194-469c-a70a-9c44dbfe04fd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:35.457203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dab18a4c-b194-469c-a70a-9c44dbfe04fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:35.538988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab18a4c-b194-469c-a70a-9c44dbfe04fd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:35.538988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:35.538988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:35.538988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:35.539555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] PATH_INFO: `/v3/users/f6830a0ff301407a89c88429241b2379` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:35.539555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6830a0ff301407a89c88429241b2379)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:35.540850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:35.560981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:35.826920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6830a0ff301407a89c88429241b2379'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:35.827568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6830a0ff301407a89c88429241b2379'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:35.834340 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6830a0ff301407a89c88429241b2379'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:35.847392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6830a0ff301407a89c88429241b2379'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:35.947915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6830a0ff301407a89c88429241b2379'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:35.954952 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1207/2382] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:01:35 2022] DELETE /identity/v3/users/f6830a0ff301407a89c88429241b2379 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:35.959503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01798b6-95ec-4535-9085-8b9aa9749540 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:35.959983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01798b6-95ec-4535-9085-8b9aa9749540 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:35.960073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01798b6-95ec-4535-9085-8b9aa9749540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:35.978935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f01798b6-95ec-4535-9085-8b9aa9749540 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:35.979573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f01798b6-95ec-4535-9085-8b9aa9749540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:36.041549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1176/2383] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:36.053050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:36.057949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:36.080703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39facb02-e23c-4a9b-ba2c-58b14320d39f None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:36.090211 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39facb02-e23c-4a9b-ba2c-58b14320d39f None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:36.090211 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39facb02-e23c-4a9b-ba2c-58b14320d39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:36.115371 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-39facb02-e23c-4a9b-ba2c-58b14320d39f None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:36.116698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-39facb02-e23c-4a9b-ba2c-58b14320d39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:36.146935 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbffbd-dd86-438d-99b3-3b0cdfe1f06a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:36.150647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ac54d-740a-4fbe-a22a-fd54531b5f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:36.151304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac54d-740a-4fbe-a22a-fd54531b5f89 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:36.151803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac54d-740a-4fbe-a22a-fd54531b5f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:36.151926 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac54d-740a-4fbe-a22a-fd54531b5f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:36.170177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ac54d-740a-4fbe-a22a-fd54531b5f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:36.171789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ac54d-740a-4fbe-a22a-fd54531b5f89 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:36.185737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1208/2384] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:01:36.191320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1177/2385] 10.222.0.42 () {58 vars in 967 bytes} [Fri Jun 3 10:01:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:01:36.207284 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ac54d-740a-4fbe-a22a-fd54531b5f89 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:36.210980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c17ac54d-740a-4fbe-a22a-fd54531b5f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:36.297364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ac54d-740a-4fbe-a22a-fd54531b5f89 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:36.302864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e318b0c-23b9-4d25-8bbb-ec9636fe10bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:36.303511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e318b0c-23b9-4d25-8bbb-ec9636fe10bc service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:36.303797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e318b0c-23b9-4d25-8bbb-ec9636fe10bc service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:36.304073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e318b0c-23b9-4d25-8bbb-ec9636fe10bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:36.313516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e318b0c-23b9-4d25-8bbb-ec9636fe10bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:36.315127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e318b0c-23b9-4d25-8bbb-ec9636fe10bc service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:36.323442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1209/2386] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:01:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:01:36.500858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39facb02-e23c-4a9b-ba2c-58b14320d39f None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:36.502515 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-39facb02-e23c-4a9b-ba2c-58b14320d39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:36.520966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39facb02-e23c-4a9b-ba2c-58b14320d39f None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:36.525869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:36.526820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:36.527105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:36.527997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] PATH_INFO: `/v3/projects/244fa4e99f7348629c9b5506f21cf21e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:36.529833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] RBAC: Authorizing `identity:delete_project(project_id=244fa4e99f7348629c9b5506f21cf21e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:36.531812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:36.748945 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244fa4e99f7348629c9b5506f21cf21e', 'request_id': 'req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:36.754491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244fa4e99f7348629c9b5506f21cf21e', 'request_id': 'req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:36.897346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244fa4e99f7348629c9b5506f21cf21e', 'request_id': 'req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:36.909529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] The token cache is being invalidate because project 244fa4e99f7348629c9b5506f21cf21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:01:36.909814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:36.910480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1d3d2a03-7cba-4fb3-a0d1-a60d428d996c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:01:36.912018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1178/2387] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:01:36 2022] DELETE /identity/v3/projects/244fa4e99f7348629c9b5506f21cf21e => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:36.917492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde535e-b816-46f0-80c1-b1cabd86f4d5 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:36.918111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde535e-b816-46f0-80c1-b1cabd86f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:36.918111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde535e-b816-46f0-80c1-b1cabd86f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:36.935538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-abde535e-b816-46f0-80c1-b1cabd86f4d5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:36.936676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-abde535e-b816-46f0-80c1-b1cabd86f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:36.953756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:37.017097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1210/2388] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:37.025426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026532e-ead9-48da-bff1-687e8901e8ce None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:37.026448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026532e-ead9-48da-bff1-687e8901e8ce None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:37.026913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026532e-ead9-48da-bff1-687e8901e8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:37.050355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c026532e-ead9-48da-bff1-687e8901e8ce None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:37.051604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c026532e-ead9-48da-bff1-687e8901e8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:37.060049 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1179/2389] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:37.064353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde535e-b816-46f0-80c1-b1cabd86f4d5 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:37.075511 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde535e-b816-46f0-80c1-b1cabd86f4d5 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:37.080368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3ea49-0210-40fa-a410-b8d62edb78cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:37.081019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3ea49-0210-40fa-a410-b8d62edb78cb admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:37.081302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3ea49-0210-40fa-a410-b8d62edb78cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:37.081577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3ea49-0210-40fa-a410-b8d62edb78cb admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:37.082357 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3ea49-0210-40fa-a410-b8d62edb78cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:01:37.082659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3ea49-0210-40fa-a410-b8d62edb78cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:37.084249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3ea49-0210-40fa-a410-b8d62edb78cb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:37.097983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1211/2390] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:01:37.101722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026532e-ead9-48da-bff1-687e8901e8ce None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:37.111353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026532e-ead9-48da-bff1-687e8901e8ce None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:37.120908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453fd5c-f990-4b36-be38-b983f06cd3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:37.121645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453fd5c-f990-4b36-be38-b983f06cd3af admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:37.121961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453fd5c-f990-4b36-be38-b983f06cd3af admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:37.122251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453fd5c-f990-4b36-be38-b983f06cd3af admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:37.122991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453fd5c-f990-4b36-be38-b983f06cd3af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:37.125036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453fd5c-f990-4b36-be38-b983f06cd3af admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:37.269933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:37.276762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1180/2391] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:01:37 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:01:37.283749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3ea49-0210-40fa-a410-b8d62edb78cb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:37.293789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3ea49-0210-40fa-a410-b8d62edb78cb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:37.350344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43831-3d87-44d3-906c-8a772cc134eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:37.351013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43831-3d87-44d3-906c-8a772cc134eb admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:37.351302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43831-3d87-44d3-906c-8a772cc134eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:37.351582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43831-3d87-44d3-906c-8a772cc134eb admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:37.352306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc43831-3d87-44d3-906c-8a772cc134eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:37.354258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc43831-3d87-44d3-906c-8a772cc134eb admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:37.486649 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1212/2392] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:01:37 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:01:37.490800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453fd5c-f990-4b36-be38-b983f06cd3af admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:37.501695 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453fd5c-f990-4b36-be38-b983f06cd3af admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:37.506291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681de7bc-7014-4e71-adee-718c4bbbc75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:37.506893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681de7bc-7014-4e71-adee-718c4bbbc75a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:37.507205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681de7bc-7014-4e71-adee-718c4bbbc75a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:37.507491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681de7bc-7014-4e71-adee-718c4bbbc75a admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:37.508306 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681de7bc-7014-4e71-adee-718c4bbbc75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:37.510584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681de7bc-7014-4e71-adee-718c4bbbc75a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:37.522503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1181/2393] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:37.531121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43831-3d87-44d3-906c-8a772cc134eb admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:37.541042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43831-3d87-44d3-906c-8a772cc134eb admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:37.545708 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d81369-d05f-4a40-883a-65f48e264555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:37.546408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81369-d05f-4a40-883a-65f48e264555 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:37.546696 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81369-d05f-4a40-883a-65f48e264555 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:37.546979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81369-d05f-4a40-883a-65f48e264555 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:37.547704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d81369-d05f-4a40-883a-65f48e264555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:37.549296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d81369-d05f-4a40-883a-65f48e264555 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:37.559912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1213/2394] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:37.563950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681de7bc-7014-4e71-adee-718c4bbbc75a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:37.574656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681de7bc-7014-4e71-adee-718c4bbbc75a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:37.579234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee9c9b-5107-4bc0-8a06-53e514efd39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:37.580108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee9c9b-5107-4bc0-8a06-53e514efd39e admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:37.580418 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee9c9b-5107-4bc0-8a06-53e514efd39e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:37.580692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee9c9b-5107-4bc0-8a06-53e514efd39e admin admin] PATH_INFO: `/v3/projects/3c44b2575d1945e0a100fe081e5af582/users/5d332241d6f84959880bfa99a24805dc/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:37.594818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee9c9b-5107-4bc0-8a06-53e514efd39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c44b2575d1945e0a100fe081e5af582, user_id=5d332241d6f84959880bfa99a24805dc, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:37.597493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee9c9b-5107-4bc0-8a06-53e514efd39e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:37.692078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1182/2395] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:01:37 2022] PUT /identity/v3/projects/3c44b2575d1945e0a100fe081e5af582/users/5d332241d6f84959880bfa99a24805dc/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:37.698520 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865f18c-f144-4396-a8fe-1ba4d87729d7 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:37.698974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865f18c-f144-4396-a8fe-1ba4d87729d7 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:37.699378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865f18c-f144-4396-a8fe-1ba4d87729d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:37.722069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-4865f18c-f144-4396-a8fe-1ba4d87729d7 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:37.723676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4865f18c-f144-4396-a8fe-1ba4d87729d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:37.789294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1214/2396] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:37.800133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee9c9b-5107-4bc0-8a06-53e514efd39e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:37.802866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-31ee9c9b-5107-4bc0-8a06-53e514efd39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:37.859583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee9c9b-5107-4bc0-8a06-53e514efd39e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:37.863519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33aba9c-5a2b-4f0a-8854-1254c56ec9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:37.864167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33aba9c-5a2b-4f0a-8854-1254c56ec9ab service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:37.864454 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33aba9c-5a2b-4f0a-8854-1254c56ec9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:37.864732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33aba9c-5a2b-4f0a-8854-1254c56ec9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:37.874662 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33aba9c-5a2b-4f0a-8854-1254c56ec9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:37.876105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33aba9c-5a2b-4f0a-8854-1254c56ec9ab service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:37.886465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1183/2397] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:01:43.975941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1d137-0d0f-4595-a402-e1e336082390 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:43.976505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1d137-0d0f-4595-a402-e1e336082390 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:43.976505 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1d137-0d0f-4595-a402-e1e336082390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:44.008398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-f4d1d137-0d0f-4595-a402-e1e336082390 None None] MFA Rules not processed for user `5d332241d6f84959880bfa99a24805dc`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:44.012225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f4d1d137-0d0f-4595-a402-e1e336082390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:44.030802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:44.110743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1215/2398] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:44.111381 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33aba9c-5a2b-4f0a-8854-1254c56ec9ab service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:44.126056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33aba9c-5a2b-4f0a-8854-1254c56ec9ab service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:44.131995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215fef-7ff6-4e11-b0ea-18d537ed3168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:44.132838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215fef-7ff6-4e11-b0ea-18d537ed3168 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:44.134034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215fef-7ff6-4e11-b0ea-18d537ed3168 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:44.134572 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215fef-7ff6-4e11-b0ea-18d537ed3168 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:44.136485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215fef-7ff6-4e11-b0ea-18d537ed3168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:44.138744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215fef-7ff6-4e11-b0ea-18d537ed3168 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:44.266137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1184/2399] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:01:44 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:01:44.270158 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1d137-0d0f-4595-a402-e1e336082390 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:44.281421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1d137-0d0f-4595-a402-e1e336082390 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:44.338963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9a2a6-f45e-44da-a0ed-bfaafabb7b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:44.339636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9a2a6-f45e-44da-a0ed-bfaafabb7b6f admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:44.339919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9a2a6-f45e-44da-a0ed-bfaafabb7b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:44.340196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9a2a6-f45e-44da-a0ed-bfaafabb7b6f admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:44.340884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9a2a6-f45e-44da-a0ed-bfaafabb7b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:44.342894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9a2a6-f45e-44da-a0ed-bfaafabb7b6f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:44.444817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1216/2400] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:01:44 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:01:44.448916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215fef-7ff6-4e11-b0ea-18d537ed3168 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:44.459390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215fef-7ff6-4e11-b0ea-18d537ed3168 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:44.465115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a739381-e12b-4a58-b642-174fdd4e3134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:44.465790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a739381-e12b-4a58-b642-174fdd4e3134 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:44.466113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a739381-e12b-4a58-b642-174fdd4e3134 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:44.466449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a739381-e12b-4a58-b642-174fdd4e3134 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:44.469323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a739381-e12b-4a58-b642-174fdd4e3134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:44.471438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a739381-e12b-4a58-b642-174fdd4e3134 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:44.489597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1185/2401] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:44.491131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9a2a6-f45e-44da-a0ed-bfaafabb7b6f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:44.511420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9a2a6-f45e-44da-a0ed-bfaafabb7b6f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:44.516540 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1df5f9-eb85-4289-971e-f9fde9939ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:44.517506 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1df5f9-eb85-4289-971e-f9fde9939ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:44.517796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1df5f9-eb85-4289-971e-f9fde9939ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:44.518115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1df5f9-eb85-4289-971e-f9fde9939ebc admin admin] PATH_INFO: `/v3/projects/c01f79bef61c457a93cd4d4ce289cf54/users/8c496c477f8a47998f0466595519acbf/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:44.545394 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1df5f9-eb85-4289-971e-f9fde9939ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c01f79bef61c457a93cd4d4ce289cf54, user_id=8c496c477f8a47998f0466595519acbf, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:44.548356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1df5f9-eb85-4289-971e-f9fde9939ebc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:44.745169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a739381-e12b-4a58-b642-174fdd4e3134 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:44.745873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1217/2402] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:01:44 2022] PUT /identity/v3/projects/c01f79bef61c457a93cd4d4ce289cf54/users/8c496c477f8a47998f0466595519acbf/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:44.772958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a739381-e12b-4a58-b642-174fdd4e3134 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:44.833205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07868b4b-0294-4dc6-bea4-cb0511d1cc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:44.833967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07868b4b-0294-4dc6-bea4-cb0511d1cc33 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:44.834249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07868b4b-0294-4dc6-bea4-cb0511d1cc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:44.834607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07868b4b-0294-4dc6-bea4-cb0511d1cc33 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:44.835365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07868b4b-0294-4dc6-bea4-cb0511d1cc33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:44.837599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07868b4b-0294-4dc6-bea4-cb0511d1cc33 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:44.850875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1186/2403] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:01:44.855166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1df5f9-eb85-4289-971e-f9fde9939ebc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:44.866644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1df5f9-eb85-4289-971e-f9fde9939ebc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:44.872816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473727d0-0be7-48be-ad1a-1df4cbde3229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:44.873872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473727d0-0be7-48be-ad1a-1df4cbde3229 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:44.874763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473727d0-0be7-48be-ad1a-1df4cbde3229 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:44.877561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473727d0-0be7-48be-ad1a-1df4cbde3229 admin admin] PATH_INFO: `/v3/projects/c01f79bef61c457a93cd4d4ce289cf54/users/8c496c477f8a47998f0466595519acbf/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:44.880355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473727d0-0be7-48be-ad1a-1df4cbde3229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c01f79bef61c457a93cd4d4ce289cf54, user_id=8c496c477f8a47998f0466595519acbf, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:44.885766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473727d0-0be7-48be-ad1a-1df4cbde3229 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:44.894999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1218/2404] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:01:44 2022] PUT /identity/v3/projects/c01f79bef61c457a93cd4d4ce289cf54/users/8c496c477f8a47998f0466595519acbf/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:01:45.954341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94b0bb-2871-4814-ba7c-238770f171a7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:45.954888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94b0bb-2871-4814-ba7c-238770f171a7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:45.957826 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94b0bb-2871-4814-ba7c-238770f171a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:45.979226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3d94b0bb-2871-4814-ba7c-238770f171a7 None None] MFA Rules not processed for user `646fecf4405c41a0bc062c6b54fd311c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:45.980736 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3d94b0bb-2871-4814-ba7c-238770f171a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:46.008192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:46.061305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1187/2405] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:51.977958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5d15b-047c-4b3b-9b14-d2bfe4c39d57 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:51.979388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5d15b-047c-4b3b-9b14-d2bfe4c39d57 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:51.979850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5d15b-047c-4b3b-9b14-d2bfe4c39d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:52.022857 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-81b5d15b-047c-4b3b-9b14-d2bfe4c39d57 None None] MFA Rules not processed for user `8c496c477f8a47998f0466595519acbf`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:52.025679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-81b5d15b-047c-4b3b-9b14-d2bfe4c39d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:52.138309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:01:52.150920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1219/2406] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:01:52.170766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94b0bb-2871-4814-ba7c-238770f171a7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:01:52.173596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3d94b0bb-2871-4814-ba7c-238770f171a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:52.251864 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94b0bb-2871-4814-ba7c-238770f171a7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:01:52.257277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14774e-0249-4dd7-9951-62a226dbe462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:01:52.257985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14774e-0249-4dd7-9951-62a226dbe462 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:52.258801 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14774e-0249-4dd7-9951-62a226dbe462 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:52.259077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14774e-0249-4dd7-9951-62a226dbe462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:52.317146 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14774e-0249-4dd7-9951-62a226dbe462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:01:52.320023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14774e-0249-4dd7-9951-62a226dbe462 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:01:52.330929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1188/2407] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:01:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:01:55.935300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f72770-2034-451b-b74d-e2c0fe64d8fa None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:01:55.935797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f72770-2034-451b-b74d-e2c0fe64d8fa None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:01:55.935974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f72770-2034-451b-b74d-e2c0fe64d8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:01:55.957395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e4f72770-2034-451b-b74d-e2c0fe64d8fa None None] MFA Rules not processed for user `5d332241d6f84959880bfa99a24805dc`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:01:55.958891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e4f72770-2034-451b-b74d-e2c0fe64d8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:01:55.967181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1220/2408] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:02:02.326055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=67 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:02:05.950178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:02:09.982071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfb351-6d7e-46cf-8c53-b4e6a978d456 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:02:09.982071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfb351-6d7e-46cf-8c53-b4e6a978d456 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:02:09.982071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfb351-6d7e-46cf-8c53-b4e6a978d456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:02:09.993202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-08dfb351-6d7e-46cf-8c53-b4e6a978d456 None None] MFA Rules not processed for user `6113d715500240098217caf641db763a`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:02:09.995134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-08dfb351-6d7e-46cf-8c53-b4e6a978d456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:02:10.049875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1189/2409] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:02:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:02:17.580338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f72770-2034-451b-b74d-e2c0fe64d8fa None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:02:17.634630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f72770-2034-451b-b74d-e2c0fe64d8fa None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:02:17.638411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9486a21-dd65-44f9-baf9-9b1088b21926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:02:17.639050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9486a21-dd65-44f9-baf9-9b1088b21926 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:02:17.639345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9486a21-dd65-44f9-baf9-9b1088b21926 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:02:17.639607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9486a21-dd65-44f9-baf9-9b1088b21926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:02:17.641386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d9486a21-dd65-44f9-baf9-9b1088b21926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:02:17.703834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9486a21-dd65-44f9-baf9-9b1088b21926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:02:17.705253 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9486a21-dd65-44f9-baf9-9b1088b21926 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:02:17.721775 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1221/2410] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:02:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:02:20.043815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=25 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:02:27.710892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=50 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:02:29.788816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90afb2-dff2-4daf-ac91-4cd23969d073 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:02:29.789297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90afb2-dff2-4daf-ac91-4cd23969d073 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:02:29.789394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90afb2-dff2-4daf-ac91-4cd23969d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:02:29.807032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-dd90afb2-dff2-4daf-ac91-4cd23969d073 None None] MFA Rules not processed for user `82621df465a640b28eb955733ceda02f`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:02:29.808226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-dd90afb2-dff2-4daf-ac91-4cd23969d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:02:29.863262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1190/2411] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:02:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:02:39.097771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9486a21-dd65-44f9-baf9-9b1088b21926 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:02:39.106491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9486a21-dd65-44f9-baf9-9b1088b21926 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:02:39.110108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b558f5c-1d28-42e5-b53c-33e57ea336e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:02:39.110809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b558f5c-1d28-42e5-b53c-33e57ea336e5 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:02:39.111098 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b558f5c-1d28-42e5-b53c-33e57ea336e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:02:39.111378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b558f5c-1d28-42e5-b53c-33e57ea336e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:02:39.121114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b558f5c-1d28-42e5-b53c-33e57ea336e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:02:39.122692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b558f5c-1d28-42e5-b53c-33e57ea336e5 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:02:39.132199 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1222/2412] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:02:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:02:39.855705 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=25 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:02:42.863636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3818-1be8-4334-87d9-b869796e6498 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:02:42.864567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3818-1be8-4334-87d9-b869796e6498 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:02:42.864883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3818-1be8-4334-87d9-b869796e6498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:02:42.904825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-223b3818-1be8-4334-87d9-b869796e6498 None None] MFA Rules not processed for user `5a304d58eff84a87953dc7af6a0afe87`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:02:42.906154 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-223b3818-1be8-4334-87d9-b869796e6498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:02:42.965338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1191/2413] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:02:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:02:49.127936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=6 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:02:52.957972 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=25 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:03:05.349825 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423096c-3da4-4306-b34a-c945c7bb41c2 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:05.352454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423096c-3da4-4306-b34a-c945c7bb41c2 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:05.352454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423096c-3da4-4306-b34a-c945c7bb41c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:05.371217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e423096c-3da4-4306-b34a-c945c7bb41c2 None None] MFA Rules not processed for user `49efae911ef8458b9288558014217934`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:03:05.372553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e423096c-3da4-4306-b34a-c945c7bb41c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:05.430792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1223/2414] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:03:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:03:12.497752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b3818-1be8-4334-87d9-b869796e6498 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:12.499938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-223b3818-1be8-4334-87d9-b869796e6498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:12.512669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b3818-1be8-4334-87d9-b869796e6498 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:12.517303 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b39a5-c60e-4dbf-b86d-997c87448bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:12.517991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b39a5-c60e-4dbf-b86d-997c87448bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:12.518281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b39a5-c60e-4dbf-b86d-997c87448bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:12.518554 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b39a5-c60e-4dbf-b86d-997c87448bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:12.520330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d86b39a5-c60e-4dbf-b86d-997c87448bf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:12.534344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b39a5-c60e-4dbf-b86d-997c87448bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:12.535813 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b39a5-c60e-4dbf-b86d-997c87448bf2 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:12.549177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1192/2415] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:12.970438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423096c-3da4-4306-b34a-c945c7bb41c2 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:12.981530 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423096c-3da4-4306-b34a-c945c7bb41c2 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:12.986156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e1f82-66a2-4666-be2c-f50908175fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:12.986855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e1f82-66a2-4666-be2c-f50908175fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:12.987138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e1f82-66a2-4666-be2c-f50908175fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:12.987414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e1f82-66a2-4666-be2c-f50908175fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:12.989227 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0f6e1f82-66a2-4666-be2c-f50908175fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:13.003242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e1f82-66a2-4666-be2c-f50908175fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:13.004892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e1f82-66a2-4666-be2c-f50908175fd1 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:13.017499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1224/2416] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:22.543273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=6 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:03:23.011895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=31 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:03:24.886657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b39a5-c60e-4dbf-b86d-997c87448bf2 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:24.895748 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b39a5-c60e-4dbf-b86d-997c87448bf2 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:24.899512 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97975985-58bc-4d8b-be82-72de081bccd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:24.900177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97975985-58bc-4d8b-be82-72de081bccd1 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:24.900618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97975985-58bc-4d8b-be82-72de081bccd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:24.901043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97975985-58bc-4d8b-be82-72de081bccd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:24.909807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-97975985-58bc-4d8b-be82-72de081bccd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:24.914504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97975985-58bc-4d8b-be82-72de081bccd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:24.915987 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97975985-58bc-4d8b-be82-72de081bccd1 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:24.925752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1193/2417] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:26.813440 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e1f82-66a2-4666-be2c-f50908175fd1 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:26.822272 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e1f82-66a2-4666-be2c-f50908175fd1 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:26.825936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:26.827216 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:26.827501 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:26.827784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:26.829694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:26.840639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:26.842169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:26.851436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1225/2418] 10.222.0.42 () {60 vars in 1396 bytes} [Fri Jun 3 10:03:26 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3242 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:33.574269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97975985-58bc-4d8b-be82-72de081bccd1 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:33.583024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97975985-58bc-4d8b-be82-72de081bccd1 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:33.586636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62cabb-bb1c-439a-9988-68dbc349ea5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:33.587273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62cabb-bb1c-439a-9988-68dbc349ea5c service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:33.587558 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62cabb-bb1c-439a-9988-68dbc349ea5c service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:33.587839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62cabb-bb1c-439a-9988-68dbc349ea5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:33.589581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8e62cabb-bb1c-439a-9988-68dbc349ea5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:33.599420 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62cabb-bb1c-439a-9988-68dbc349ea5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:33.600842 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62cabb-bb1c-439a-9988-68dbc349ea5c service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:33.608734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1194/2419] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:03:34.997298 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:34.998845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:35.015016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e7101-5a0d-4045-80ec-c50823b7f6b4 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:35.020994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:35.021711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:35.022030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:35.022766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:35.024814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:35.039990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:35.041634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:35.053924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1226/2420] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:36.872811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62cabb-bb1c-439a-9988-68dbc349ea5c service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:36.882098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62cabb-bb1c-439a-9988-68dbc349ea5c service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:36.885982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d602fc-7a92-4ad9-b260-111ef2f79943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:36.886610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d602fc-7a92-4ad9-b260-111ef2f79943 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:36.886916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d602fc-7a92-4ad9-b260-111ef2f79943 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:36.887192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d602fc-7a92-4ad9-b260-111ef2f79943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:36.889055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-88d602fc-7a92-4ad9-b260-111ef2f79943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:36.950876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d602fc-7a92-4ad9-b260-111ef2f79943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:36.952236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d602fc-7a92-4ad9-b260-111ef2f79943 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:36.961762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1195/2421] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:40.658595 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:40.660245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:40.675493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126bb0-7f1b-47c5-8aed-4e0d6d4324a2 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:40.680332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:40.681413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:40.681698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:40.682031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] PATH_INFO: `/v3/users/6113d715500240098217caf641db763a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:40.683217 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6113d715500240098217caf641db763a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:40.688912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:40.897699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6113d715500240098217caf641db763a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:40.898390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6113d715500240098217caf641db763a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:40.905153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6113d715500240098217caf641db763a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:40.917209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6113d715500240098217caf641db763a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:41.020944 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-32dc3be8-262e-418f-bdd7-b881945c12f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6113d715500240098217caf641db763a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:41.027319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1227/2422] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:03:40 2022] DELETE /identity/v3/users/6113d715500240098217caf641db763a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:03:41.307277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d602fc-7a92-4ad9-b260-111ef2f79943 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:41.318193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d602fc-7a92-4ad9-b260-111ef2f79943 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:41.392387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:41.393830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:41.394307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:41.395097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] PATH_INFO: `/v3/projects/071b399988474af1a1c709aac97c1744` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:41.396793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=071b399988474af1a1c709aac97c1744)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:41.398760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:41.535763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071b399988474af1a1c709aac97c1744', 'request_id': 'req-5914e6cd-31dc-45b4-a457-45565cb56c44'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:41.541464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071b399988474af1a1c709aac97c1744', 'request_id': 'req-5914e6cd-31dc-45b4-a457-45565cb56c44'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:41.620815 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071b399988474af1a1c709aac97c1744', 'request_id': 'req-5914e6cd-31dc-45b4-a457-45565cb56c44'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:41.629852 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:03:41.635039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] The token cache is being invalidate because project 071b399988474af1a1c709aac97c1744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:03:41.647035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:41.647035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5914e6cd-31dc-45b4-a457-45565cb56c44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:41.647035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1196/2423] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:03:41 2022] DELETE /identity/v3/projects/071b399988474af1a1c709aac97c1744 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:03:41.648716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23777b5a-c534-4282-8a5d-b8d770abbde6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:41.649015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23777b5a-c534-4282-8a5d-b8d770abbde6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:41.649418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23777b5a-c534-4282-8a5d-b8d770abbde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:41.668223 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-23777b5a-c534-4282-8a5d-b8d770abbde6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:03:41.669407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-23777b5a-c534-4282-8a5d-b8d770abbde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:41.728242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1228/2424] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:03:41.736882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9d02b-e821-477b-8ec8-b608d170fd74 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:41.737176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9d02b-e821-477b-8ec8-b608d170fd74 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:41.737487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9d02b-e821-477b-8ec8-b608d170fd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:41.756136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a2a9d02b-e821-477b-8ec8-b608d170fd74 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:03:41.757341 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a2a9d02b-e821-477b-8ec8-b608d170fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:41.765657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1197/2425] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:03:41.769810 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23777b5a-c534-4282-8a5d-b8d770abbde6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:41.779544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23777b5a-c534-4282-8a5d-b8d770abbde6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:41.784626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b80af-aabb-4b27-8c33-63d1121241a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:41.785307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b80af-aabb-4b27-8c33-63d1121241a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:41.785592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b80af-aabb-4b27-8c33-63d1121241a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:41.785871 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b80af-aabb-4b27-8c33-63d1121241a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:41.786668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b80af-aabb-4b27-8c33-63d1121241a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:03:41.792107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b80af-aabb-4b27-8c33-63d1121241a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:41.792107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b80af-aabb-4b27-8c33-63d1121241a4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:41.802047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1229/2426] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:03:41.806047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9d02b-e821-477b-8ec8-b608d170fd74 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:41.815580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9d02b-e821-477b-8ec8-b608d170fd74 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:41.820281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed20cd0-63fa-4fd1-972c-2364bb096a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:41.820916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed20cd0-63fa-4fd1-972c-2364bb096a81 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:41.821198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed20cd0-63fa-4fd1-972c-2364bb096a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:41.821500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed20cd0-63fa-4fd1-972c-2364bb096a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:41.822248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed20cd0-63fa-4fd1-972c-2364bb096a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:41.824251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed20cd0-63fa-4fd1-972c-2364bb096a81 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:41.947947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1198/2427] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:03:41 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:03:41.951960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b80af-aabb-4b27-8c33-63d1121241a4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:41.962429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b80af-aabb-4b27-8c33-63d1121241a4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:41.993084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:03:42.019690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330106b-b9e6-4e07-8c59-39c6216c0da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:42.020450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330106b-b9e6-4e07-8c59-39c6216c0da8 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:42.020735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330106b-b9e6-4e07-8c59-39c6216c0da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:42.021016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330106b-b9e6-4e07-8c59-39c6216c0da8 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:42.021714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330106b-b9e6-4e07-8c59-39c6216c0da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:42.023691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330106b-b9e6-4e07-8c59-39c6216c0da8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:42.147583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1230/2428] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:03:41 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:03:42.151710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed20cd0-63fa-4fd1-972c-2364bb096a81 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:42.162722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed20cd0-63fa-4fd1-972c-2364bb096a81 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:42.167485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b94eb-2a87-426c-9541-9038e2547658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:42.168142 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b94eb-2a87-426c-9541-9038e2547658 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:42.168429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b94eb-2a87-426c-9541-9038e2547658 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:42.168709 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b94eb-2a87-426c-9541-9038e2547658 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:42.169475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b94eb-2a87-426c-9541-9038e2547658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:42.171111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b94eb-2a87-426c-9541-9038e2547658 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:42.181654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1199/2429] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:03:42.185801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330106b-b9e6-4e07-8c59-39c6216c0da8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:42.196061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330106b-b9e6-4e07-8c59-39c6216c0da8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:42.200811 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75147c1d-c1af-460e-85b4-2977580dded5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:42.201537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75147c1d-c1af-460e-85b4-2977580dded5 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:42.201837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75147c1d-c1af-460e-85b4-2977580dded5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:42.202374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75147c1d-c1af-460e-85b4-2977580dded5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:42.203280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75147c1d-c1af-460e-85b4-2977580dded5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:42.206205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75147c1d-c1af-460e-85b4-2977580dded5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:42.216632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1231/2430] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:03:42.221346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b94eb-2a87-426c-9541-9038e2547658 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:42.231484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b94eb-2a87-426c-9541-9038e2547658 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:42.236212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b4f7e-585f-42ca-becf-3ea6cfffcead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:42.237115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b4f7e-585f-42ca-becf-3ea6cfffcead admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:42.237402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b4f7e-585f-42ca-becf-3ea6cfffcead admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:42.237683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b4f7e-585f-42ca-becf-3ea6cfffcead admin admin] PATH_INFO: `/v3/projects/3e51010dd5d84daf8aa43859e9e7cb6d/users/ce77221a2c8d4c4395bed092c1967b28/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:42.253288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b4f7e-585f-42ca-becf-3ea6cfffcead admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e51010dd5d84daf8aa43859e9e7cb6d, user_id=ce77221a2c8d4c4395bed092c1967b28, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:42.257709 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b4f7e-585f-42ca-becf-3ea6cfffcead admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:42.358640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1200/2431] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:03:42 2022] PUT /identity/v3/projects/3e51010dd5d84daf8aa43859e9e7cb6d/users/ce77221a2c8d4c4395bed092c1967b28/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:03:42.364225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06362b78-69ae-46e7-bb17-2364b25abd17 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:42.364519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06362b78-69ae-46e7-bb17-2364b25abd17 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:42.364803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06362b78-69ae-46e7-bb17-2364b25abd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:42.382932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-06362b78-69ae-46e7-bb17-2364b25abd17 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:03:42.384390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-06362b78-69ae-46e7-bb17-2364b25abd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:42.444173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1232/2432] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:03:42.459225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b4f7e-585f-42ca-becf-3ea6cfffcead admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:42.459225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3a4b4f7e-585f-42ca-becf-3ea6cfffcead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:42.511368 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b4f7e-585f-42ca-becf-3ea6cfffcead admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:42.514975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468630a-c903-401e-9095-d1ca197b3c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:42.515605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468630a-c903-401e-9095-d1ca197b3c89 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:42.515884 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468630a-c903-401e-9095-d1ca197b3c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:42.516159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468630a-c903-401e-9095-d1ca197b3c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:42.524787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468630a-c903-401e-9095-d1ca197b3c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:42.526230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468630a-c903-401e-9095-d1ca197b3c89 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:42.535833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1201/2433] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:46.009834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06362b78-69ae-46e7-bb17-2364b25abd17 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:46.010241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-06362b78-69ae-46e7-bb17-2364b25abd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:46.129507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06362b78-69ae-46e7-bb17-2364b25abd17 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:46.136680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:46.137473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:46.137932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:46.138238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:46.140968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:46.230094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:03:46.243604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:46.245792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:46.255484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652898e-e0c8-40c0-b32c-657f3312b127 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:46.256400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652898e-e0c8-40c0-b32c-657f3312b127 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:46.256827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652898e-e0c8-40c0-b32c-657f3312b127 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:46.258404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1202/2434] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 10:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 10:03:46.264289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbac29-557d-4a84-b188-68575145e1bb None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:46.264786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbac29-557d-4a84-b188-68575145e1bb None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:46.265272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbac29-557d-4a84-b188-68575145e1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:46.271771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1233/2435] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:46.340332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e0dbac29-557d-4a84-b188-68575145e1bb None None] MFA Rules not processed for user `566c0c4bdcaf4c13b060af4bbff5bc04`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:03:46.344676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e0dbac29-557d-4a84-b188-68575145e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:46.385308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:46.386927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:46.408587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874cd0da-8a01-4955-baff-59b342c04613 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:46.415134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:46.416187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:46.416545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:46.416900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] PATH_INFO: `/v3/users/82621df465a640b28eb955733ceda02f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:46.418753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82621df465a640b28eb955733ceda02f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:46.426172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:46.500998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1203/2436] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 10:03:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:03:46.512904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbac29-557d-4a84-b188-68575145e1bb None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:46.514690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e0dbac29-557d-4a84-b188-68575145e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:46.529234 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbac29-557d-4a84-b188-68575145e1bb None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:46.537279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b68b0f-ea3f-4066-bd0b-c00d841f9c9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:46.538304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b68b0f-ea3f-4066-bd0b-c00d841f9c9b service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:46.538601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b68b0f-ea3f-4066-bd0b-c00d841f9c9b service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:46.538885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b68b0f-ea3f-4066-bd0b-c00d841f9c9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:46.549470 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b68b0f-ea3f-4066-bd0b-c00d841f9c9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:46.552630 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b68b0f-ea3f-4066-bd0b-c00d841f9c9b service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:46.562057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1204/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:03:46.760523 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82621df465a640b28eb955733ceda02f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:46.761197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82621df465a640b28eb955733ceda02f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:46.768829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82621df465a640b28eb955733ceda02f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:46.782008 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82621df465a640b28eb955733ceda02f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:46.855760 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2f8fbbfb-a49f-4b6f-a172-359208b4e6b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82621df465a640b28eb955733ceda02f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:46.863580 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1234/2438] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:03:46 2022] DELETE /identity/v3/users/82621df465a640b28eb955733ceda02f => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:03:47.162347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b68b0f-ea3f-4066-bd0b-c00d841f9c9b service placement] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:47.181498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b68b0f-ea3f-4066-bd0b-c00d841f9c9b service placement] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:47.185546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:03:47.285281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:47.287113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:47.287560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:47.288069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] PATH_INFO: `/v3/projects/6a0f8d737269433bb6ce1baf51b8157b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:47.289672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a0f8d737269433bb6ce1baf51b8157b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:47.292882 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:47.442823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a0f8d737269433bb6ce1baf51b8157b', 'request_id': 'req-9ebc39fb-b924-475f-80f4-433cd0eec293'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:47.450948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a0f8d737269433bb6ce1baf51b8157b', 'request_id': 'req-9ebc39fb-b924-475f-80f4-433cd0eec293'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:47.519399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a0f8d737269433bb6ce1baf51b8157b', 'request_id': 'req-9ebc39fb-b924-475f-80f4-433cd0eec293'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:47.533854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] The token cache is being invalidate because project 6a0f8d737269433bb6ce1baf51b8157b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:03:47.534286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:47.535620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:03:47.543446 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1205/2439] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:03:47 2022] DELETE /identity/v3/projects/6a0f8d737269433bb6ce1baf51b8157b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:03:48.395753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a43d03-b253-4fce-ac93-e08c1e1fcbdd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:48.396236 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a43d03-b253-4fce-ac93-e08c1e1fcbdd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:48.396354 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a43d03-b253-4fce-ac93-e08c1e1fcbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:48.424738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a5a43d03-b253-4fce-ac93-e08c1e1fcbdd None None] MFA Rules not processed for user `ce77221a2c8d4c4395bed092c1967b28`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:03:48.425863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a5a43d03-b253-4fce-ac93-e08c1e1fcbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:48.505827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1235/2440] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:03:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:03:48.522827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:48.524308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:48.610074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc39fb-b924-475f-80f4-433cd0eec293 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:48.615606 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158870c0-9679-4305-a0ff-0252b4549b45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:48.616255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158870c0-9679-4305-a0ff-0252b4549b45 service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:48.616585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158870c0-9679-4305-a0ff-0252b4549b45 service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:48.616868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158870c0-9679-4305-a0ff-0252b4549b45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:48.625670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158870c0-9679-4305-a0ff-0252b4549b45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:48.628004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158870c0-9679-4305-a0ff-0252b4549b45 service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:48.637784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1206/2441] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:51.185294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99143-2284-4857-8be9-ef0d165d76db None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:51.185816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99143-2284-4857-8be9-ef0d165d76db None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:51.185965 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c99143-2284-4857-8be9-ef0d165d76db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:51.206764 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a2c99143-2284-4857-8be9-ef0d165d76db None None] MFA Rules not processed for user `ce77221a2c8d4c4395bed092c1967b28`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:03:51.214011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a2c99143-2284-4857-8be9-ef0d165d76db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:51.222160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1236/2442] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:03:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:03:51.617067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158870c0-9679-4305-a0ff-0252b4549b45 service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:51.620375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-158870c0-9679-4305-a0ff-0252b4549b45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:51.662494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:03:51.709963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158870c0-9679-4305-a0ff-0252b4549b45 service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:51.721262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfa0e2-ad22-45d2-bcd2-83d1e9f67057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:51.722359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfa0e2-ad22-45d2-bcd2-83d1e9f67057 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:51.723023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfa0e2-ad22-45d2-bcd2-83d1e9f67057 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:51.724179 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfa0e2-ad22-45d2-bcd2-83d1e9f67057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:51.727355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-18cfa0e2-ad22-45d2-bcd2-83d1e9f67057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:51.742381 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfa0e2-ad22-45d2-bcd2-83d1e9f67057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:51.744944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfa0e2-ad22-45d2-bcd2-83d1e9f67057 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:51.756293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1207/2443] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:03:52.206416 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c99143-2284-4857-8be9-ef0d165d76db None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:03:52.207875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a2c99143-2284-4857-8be9-ef0d165d76db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:52.260048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:03:52.283667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c99143-2284-4857-8be9-ef0d165d76db None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:03:52.288789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c52efb-ff91-47dc-b2e8-5cf86c808238 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:03:52.289682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c52efb-ff91-47dc-b2e8-5cf86c808238 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:03:52.290388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c52efb-ff91-47dc-b2e8-5cf86c808238 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:03:52.290714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c52efb-ff91-47dc-b2e8-5cf86c808238 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:03:52.292517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e6c52efb-ff91-47dc-b2e8-5cf86c808238 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:03:52.304244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c52efb-ff91-47dc-b2e8-5cf86c808238 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:03:52.305854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c52efb-ff91-47dc-b2e8-5cf86c808238 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:03:52.316020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1237/2444] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:03:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:04:01.209346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfa0e2-ad22-45d2-bcd2-83d1e9f67057 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:01.226196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfa0e2-ad22-45d2-bcd2-83d1e9f67057 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:01.231888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:01.232546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:01.232828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:01.233102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:01.236274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:01.252018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:01.254071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:01.266698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1208/2445] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:04:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:04:02.352085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=12 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:04:02.526552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6114ba-a4e5-4fc2-8b8a-34956abe9e11 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:02.526839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6114ba-a4e5-4fc2-8b8a-34956abe9e11 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:02.527145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6114ba-a4e5-4fc2-8b8a-34956abe9e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:02.545466 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-bc6114ba-a4e5-4fc2-8b8a-34956abe9e11 None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:02.546800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bc6114ba-a4e5-4fc2-8b8a-34956abe9e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:02.601637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1238/2446] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 10:04:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:02.779645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:02.781057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:02.848934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087e218-b4de-4939-a70d-10b3600540af service nova] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:02.854001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099f140-fd25-4148-9ef3-75a6e527dd92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:02.854646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f140-fd25-4148-9ef3-75a6e527dd92 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:02.854933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f140-fd25-4148-9ef3-75a6e527dd92 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:02.855216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f140-fd25-4148-9ef3-75a6e527dd92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:02.865686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099f140-fd25-4148-9ef3-75a6e527dd92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:02.865686 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099f140-fd25-4148-9ef3-75a6e527dd92 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:02.873167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1209/2447] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:04:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:04:12.594578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=25 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:04:12.869689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=54 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:04:20.669614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6114ba-a4e5-4fc2-8b8a-34956abe9e11 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:20.679404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6114ba-a4e5-4fc2-8b8a-34956abe9e11 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:20.682994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34bf2d-b41d-4e47-8f93-46b436aaf491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:20.683629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34bf2d-b41d-4e47-8f93-46b436aaf491 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:20.683914 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34bf2d-b41d-4e47-8f93-46b436aaf491 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:20.684192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34bf2d-b41d-4e47-8f93-46b436aaf491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:20.686003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8c34bf2d-b41d-4e47-8f93-46b436aaf491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:20.695334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34bf2d-b41d-4e47-8f93-46b436aaf491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:20.696772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34bf2d-b41d-4e47-8f93-46b436aaf491 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:20.704828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1239/2448] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:04:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:04:20.966031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d4d35-f6d3-45eb-b1a6-c588a604ebc3 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:20.966617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d4d35-f6d3-45eb-b1a6-c588a604ebc3 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:20.966617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d4d35-f6d3-45eb-b1a6-c588a604ebc3 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:20.968030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1210/2449] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 10:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 10:04:20.975134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d9266-f929-485e-9aa1-06365b2921cc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:20.975428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d9266-f929-485e-9aa1-06365b2921cc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:20.975710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d9266-f929-485e-9aa1-06365b2921cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:20.994017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2e0d9266-f929-485e-9aa1-06365b2921cc None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:20.995224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2e0d9266-f929-485e-9aa1-06365b2921cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:21.004605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1240/2450] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 10:04:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:21.016756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d4d35-f6d3-45eb-b1a6-c588a604ebc3 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:21.019013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bc8d4d35-f6d3-45eb-b1a6-c588a604ebc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:21.099742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d4d35-f6d3-45eb-b1a6-c588a604ebc3 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:21.103304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c4b54-e652-45f7-afb0-573ec4de8154 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd59a351971e94aa38a22e8113207e5b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:21.103944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c4b54-e652-45f7-afb0-573ec4de8154 service glance] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:21.104230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c4b54-e652-45f7-afb0-573ec4de8154 service glance] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:21.104507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c4b54-e652-45f7-afb0-573ec4de8154 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:21.114487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c4b54-e652-45f7-afb0-573ec4de8154 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:21.115920 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c4b54-e652-45f7-afb0-573ec4de8154 service glance] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:21.126469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1211/2451] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:04:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:04:21.347013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ec31f-f743-4cf4-8de6-1c197a787f56 None None] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:21.347325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ec31f-f743-4cf4-8de6-1c197a787f56 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:21.347605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ec31f-f743-4cf4-8de6-1c197a787f56 None None] PATH_INFO: `/` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:21.348992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1241/2452] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 10:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 10:04:21.353698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962ddf7-e696-44c4-af72-59caa7f72574 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:21.356287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962ddf7-e696-44c4-af72-59caa7f72574 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:21.356287 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962ddf7-e696-44c4-af72-59caa7f72574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:21.372170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-6962ddf7-e696-44c4-af72-59caa7f72574 None None] MFA Rules not processed for user `362797538cbd48dc9c73b09b0548959e`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:21.373372 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6962ddf7-e696-44c4-af72-59caa7f72574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:21.380260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1212/2453] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:21.391921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ec31f-f743-4cf4-8de6-1c197a787f56 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:21.404028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ec31f-f743-4cf4-8de6-1c197a787f56 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:21.407508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc931dea-c243-4f6c-b98e-807c1392a1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:21.408172 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931dea-c243-4f6c-b98e-807c1392a1ad service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:21.408457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931dea-c243-4f6c-b98e-807c1392a1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:21.408739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931dea-c243-4f6c-b98e-807c1392a1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:21.415883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc931dea-c243-4f6c-b98e-807c1392a1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:21.417372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc931dea-c243-4f6c-b98e-807c1392a1ad service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:21.425233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1242/2454] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:04:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:04:21.507609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60c201-52c4-437b-b093-44c44ec2bcc5 None None] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:21.507901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60c201-52c4-437b-b093-44c44ec2bcc5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:21.508221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60c201-52c4-437b-b093-44c44ec2bcc5 None None] PATH_INFO: `/` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:21.509867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1213/2455] 10.222.0.42 () {54 vars in 921 bytes} [Fri Jun 3 10:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 10:04:21.515348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d0d6f-d78d-4e3f-96c1-d3fbe74ad5b1 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:21.515636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d0d6f-d78d-4e3f-96c1-d3fbe74ad5b1 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:21.515915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d0d6f-d78d-4e3f-96c1-d3fbe74ad5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:21.534791 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-da3d0d6f-d78d-4e3f-96c1-d3fbe74ad5b1 None None] MFA Rules not processed for user `dfe8ef12db914a9693ef470f1706bb5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:21.535997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-da3d0d6f-d78d-4e3f-96c1-d3fbe74ad5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:21.545459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1243/2456] 10.222.0.42 () {58 vars in 1018 bytes} [Fri Jun 3 10:04:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:21.555951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60c201-52c4-437b-b093-44c44ec2bcc5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:21.567564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60c201-52c4-437b-b093-44c44ec2bcc5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:21.571094 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b78ad-3130-4004-8058-a000c3e191d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:21.571721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b78ad-3130-4004-8058-a000c3e191d3 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:21.572008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b78ad-3130-4004-8058-a000c3e191d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:21.572286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b78ad-3130-4004-8058-a000c3e191d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:21.583308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b78ad-3130-4004-8058-a000c3e191d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:21.584725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b78ad-3130-4004-8058-a000c3e191d3 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:21.595436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1214/2457] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:04:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:04:23.694884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e575d-cbfe-40ea-b070-464dce47865f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:23.695342 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e575d-cbfe-40ea-b070-464dce47865f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:23.695450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e575d-cbfe-40ea-b070-464dce47865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:23.714183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-216e575d-cbfe-40ea-b070-464dce47865f None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:23.715628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-216e575d-cbfe-40ea-b070-464dce47865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:23.726806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1244/2458] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 10:04:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:23.740088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b78ad-3130-4004-8058-a000c3e191d3 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:23.751370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b78ad-3130-4004-8058-a000c3e191d3 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:23.756394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca394154-cfa2-4b88-b3a0-36477ad6041a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:23.757030 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca394154-cfa2-4b88-b3a0-36477ad6041a service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:23.757315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca394154-cfa2-4b88-b3a0-36477ad6041a service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:23.757596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca394154-cfa2-4b88-b3a0-36477ad6041a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:23.765908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca394154-cfa2-4b88-b3a0-36477ad6041a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:23.767530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca394154-cfa2-4b88-b3a0-36477ad6041a service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:23.775540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1215/2459] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:04:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:04:33.707750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=21 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:04:33.772261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=50 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:04:39.368486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807c6af-5fe6-44e1-9c1e-cc68b3dd944a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:39.368980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807c6af-5fe6-44e1-9c1e-cc68b3dd944a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:39.369088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807c6af-5fe6-44e1-9c1e-cc68b3dd944a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:39.391137 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-3807c6af-5fe6-44e1-9c1e-cc68b3dd944a None None] MFA Rules not processed for user `646fecf4405c41a0bc062c6b54fd311c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:39.392800 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3807c6af-5fe6-44e1-9c1e-cc68b3dd944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:39.456855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1245/2460] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:04:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:40.849535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0030b0-c464-4074-b077-5c2da440d9a7 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:40.850068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0030b0-c464-4074-b077-5c2da440d9a7 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:40.850198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0030b0-c464-4074-b077-5c2da440d9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:40.868090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8b0030b0-c464-4074-b077-5c2da440d9a7 None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:40.869286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8b0030b0-c464-4074-b077-5c2da440d9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:40.876584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1216/2461] 10.222.0.42 () {58 vars in 967 bytes} [Fri Jun 3 10:04:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:40.889576 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807c6af-5fe6-44e1-9c1e-cc68b3dd944a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:40.902436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807c6af-5fe6-44e1-9c1e-cc68b3dd944a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:40.907373 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a411543-1ca8-406f-b877-381b9a6b4618 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:40.908015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a411543-1ca8-406f-b877-381b9a6b4618 service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:40.908332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a411543-1ca8-406f-b877-381b9a6b4618 service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:40.908636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a411543-1ca8-406f-b877-381b9a6b4618 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:40.917621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a411543-1ca8-406f-b877-381b9a6b4618 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:40.919327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a411543-1ca8-406f-b877-381b9a6b4618 service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:40.927828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1246/2462] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:04:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:04:50.862764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:04:50.924808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=31 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:04:52.746635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0030b0-c464-4074-b077-5c2da440d9a7 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:52.748130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8b0030b0-c464-4074-b077-5c2da440d9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:52.768002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0030b0-c464-4074-b077-5c2da440d9a7 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:52.768002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:52.768002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:52.768002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:52.768541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:52.768541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:52.842761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:52.844278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:52.855034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1217/2463] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:04:55.550895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f7eac-a922-464f-a2ab-6029eb4e6102 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:55.555474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f7eac-a922-464f-a2ab-6029eb4e6102 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:55.555998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f7eac-a922-464f-a2ab-6029eb4e6102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:55.579707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-ce5f7eac-a922-464f-a2ab-6029eb4e6102 None None] MFA Rules not processed for user `ce77221a2c8d4c4395bed092c1967b28`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:55.582127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ce5f7eac-a922-464f-a2ab-6029eb4e6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:55.600553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1247/2464] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:04:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:56.933073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:56.934534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:56.950069 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10235d-8c65-4831-8f03-e1c8f1b819ab service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:56.955057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e57fc-00d9-41cc-a9c0-83a7d0fd88aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:56.956351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e57fc-00d9-41cc-a9c0-83a7d0fd88aa service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:56.958065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e57fc-00d9-41cc-a9c0-83a7d0fd88aa service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:56.958065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e57fc-00d9-41cc-a9c0-83a7d0fd88aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:56.960724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-214e57fc-00d9-41cc-a9c0-83a7d0fd88aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:56.974684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e57fc-00d9-41cc-a9c0-83a7d0fd88aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:56.976232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e57fc-00d9-41cc-a9c0-83a7d0fd88aa service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:56.987421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1218/2465] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:04:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:04:57.050189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f7eac-a922-464f-a2ab-6029eb4e6102 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:57.051868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ce5f7eac-a922-464f-a2ab-6029eb4e6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:57.091974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f7eac-a922-464f-a2ab-6029eb4e6102 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:57.091974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:57.091974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:57.091974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:57.092469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] PATH_INFO: `/v3/users/49efae911ef8458b9288558014217934` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:57.092469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49efae911ef8458b9288558014217934)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:57.092469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:57.307379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49efae911ef8458b9288558014217934'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:57.308044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49efae911ef8458b9288558014217934'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:57.315291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49efae911ef8458b9288558014217934'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:57.327411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49efae911ef8458b9288558014217934'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:57.444122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dd963ae0-7420-4f69-8f24-846f3efa7227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49efae911ef8458b9288558014217934'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:57.452644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1248/2466] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:04:57 2022] DELETE /identity/v3/users/49efae911ef8458b9288558014217934 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:04:57.763144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e57fc-00d9-41cc-a9c0-83a7d0fd88aa service placement] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:57.776178 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e57fc-00d9-41cc-a9c0-83a7d0fd88aa service placement] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:57.872934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:57.873974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:57.874268 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:57.874765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] PATH_INFO: `/v3/projects/686f552be5824c8aa31930a6e8844255` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:57.876433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=686f552be5824c8aa31930a6e8844255)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:57.879270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:57.995088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '686f552be5824c8aa31930a6e8844255', 'request_id': 'req-7821af51-0210-4570-98f9-cc510daeb297'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:58.000587 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '686f552be5824c8aa31930a6e8844255', 'request_id': 'req-7821af51-0210-4570-98f9-cc510daeb297'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:58.105970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '686f552be5824c8aa31930a6e8844255', 'request_id': 'req-7821af51-0210-4570-98f9-cc510daeb297'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:58.118533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] The token cache is being invalidate because project 686f552be5824c8aa31930a6e8844255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:04:58.118816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:58.119790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-7821af51-0210-4570-98f9-cc510daeb297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:04:58.121763 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1219/2467] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:04:57 2022] DELETE /identity/v3/projects/686f552be5824c8aa31930a6e8844255 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:04:58.126749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf33331-c62e-4145-887a-6d4b87a5c7e4 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.127234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf33331-c62e-4145-887a-6d4b87a5c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.127346 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf33331-c62e-4145-887a-6d4b87a5c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.148293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1cf33331-c62e-4145-887a-6d4b87a5c7e4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:58.149611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1cf33331-c62e-4145-887a-6d4b87a5c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:58.220306 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1249/2468] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:58.229169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd6629-3ed4-4645-b39a-ce214ee14920 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.229471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd6629-3ed4-4645-b39a-ce214ee14920 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.230027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd6629-3ed4-4645-b39a-ce214ee14920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.248963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-fedd6629-3ed4-4645-b39a-ce214ee14920 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:58.250208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fedd6629-3ed4-4645-b39a-ce214ee14920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:58.258525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1220/2469] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:58.262671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf33331-c62e-4145-887a-6d4b87a5c7e4 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:58.272296 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf33331-c62e-4145-887a-6d4b87a5c7e4 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:58.276785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc8055-638e-4c03-a535-78e96879f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:58.277484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc8055-638e-4c03-a535-78e96879f573 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.277704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc8055-638e-4c03-a535-78e96879f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.278040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc8055-638e-4c03-a535-78e96879f573 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.278765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc8055-638e-4c03-a535-78e96879f573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:04:58.279055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc8055-638e-4c03-a535-78e96879f573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:58.280625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc8055-638e-4c03-a535-78e96879f573 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:58.298854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1250/2470] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:04:58.299784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd6629-3ed4-4645-b39a-ce214ee14920 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:58.302960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:04:58.306844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd6629-3ed4-4645-b39a-ce214ee14920 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:58.311469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ffb93-2c0a-4ced-9dbc-4704cdd0b8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:58.312118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ffb93-2c0a-4ced-9dbc-4704cdd0b8fb admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.312402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ffb93-2c0a-4ced-9dbc-4704cdd0b8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.312693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ffb93-2c0a-4ced-9dbc-4704cdd0b8fb admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.313354 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ffb93-2c0a-4ced-9dbc-4704cdd0b8fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:58.315360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ffb93-2c0a-4ced-9dbc-4704cdd0b8fb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:58.401779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1221/2471] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:04:58 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:04:58.403227 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc8055-638e-4c03-a535-78e96879f573 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:58.414078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc8055-638e-4c03-a535-78e96879f573 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:58.464542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:04:58.482213 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5c5ca-2658-41ae-ab41-0718c5424248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:58.482890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c5ca-2658-41ae-ab41-0718c5424248 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.483179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c5ca-2658-41ae-ab41-0718c5424248 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.483467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c5ca-2658-41ae-ab41-0718c5424248 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.484206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5c5ca-2658-41ae-ab41-0718c5424248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:58.486160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5c5ca-2658-41ae-ab41-0718c5424248 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:58.563724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1251/2472] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:04:58 2022] POST /identity/v3/users => generated 282 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:04:58.567866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ffb93-2c0a-4ced-9dbc-4704cdd0b8fb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:58.578536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ffb93-2c0a-4ced-9dbc-4704cdd0b8fb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:58.583934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce5b93-d305-4fb2-8bcc-84ab277e9292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:58.584608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce5b93-d305-4fb2-8bcc-84ab277e9292 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.584925 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce5b93-d305-4fb2-8bcc-84ab277e9292 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.585205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce5b93-d305-4fb2-8bcc-84ab277e9292 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.585958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce5b93-d305-4fb2-8bcc-84ab277e9292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:58.587578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce5b93-d305-4fb2-8bcc-84ab277e9292 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:58.599856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1222/2473] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:04:58.603994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5c5ca-2658-41ae-ab41-0718c5424248 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:58.614084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5c5ca-2658-41ae-ab41-0718c5424248 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:58.618725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeff6cc-232f-4046-ae00-e862c67448b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:58.619406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeff6cc-232f-4046-ae00-e862c67448b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.619669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeff6cc-232f-4046-ae00-e862c67448b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.619947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeff6cc-232f-4046-ae00-e862c67448b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.620702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeff6cc-232f-4046-ae00-e862c67448b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:58.622347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeff6cc-232f-4046-ae00-e862c67448b4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:58.633378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1252/2474] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:04:58.637636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce5b93-d305-4fb2-8bcc-84ab277e9292 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:58.648050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce5b93-d305-4fb2-8bcc-84ab277e9292 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:58.652670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe5a65-9cd5-49ee-9f26-127319be509f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:58.653546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe5a65-9cd5-49ee-9f26-127319be509f admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.653829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe5a65-9cd5-49ee-9f26-127319be509f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.654144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe5a65-9cd5-49ee-9f26-127319be509f admin admin] PATH_INFO: `/v3/projects/d5fef7f41c2e413fb887cabe80e25e10/users/f07c695b4137479da5998cae51bea565/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.669343 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe5a65-9cd5-49ee-9f26-127319be509f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5fef7f41c2e413fb887cabe80e25e10, user_id=f07c695b4137479da5998cae51bea565, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:58.672218 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe5a65-9cd5-49ee-9f26-127319be509f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:58.797597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1223/2475] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:04:58 2022] PUT /identity/v3/projects/d5fef7f41c2e413fb887cabe80e25e10/users/f07c695b4137479da5998cae51bea565/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:04:58.799794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ac03f-7ed5-43bc-87c4-4754f7e8c21e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.800083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ac03f-7ed5-43bc-87c4-4754f7e8c21e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.800360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ac03f-7ed5-43bc-87c4-4754f7e8c21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.835294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-870ac03f-7ed5-43bc-87c4-4754f7e8c21e None None] MFA Rules not processed for user `f07c695b4137479da5998cae51bea565`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:04:58.836671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-870ac03f-7ed5-43bc-87c4-4754f7e8c21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:58.901042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1253/2476] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:04:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:04:58.912948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe5a65-9cd5-49ee-9f26-127319be509f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:04:58.914741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6cbe5a65-9cd5-49ee-9f26-127319be509f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:04:58.971855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe5a65-9cd5-49ee-9f26-127319be509f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:04:58.975475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22994a18-5101-4c8d-949f-b1a6b639a6ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:04:58.976130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22994a18-5101-4c8d-949f-b1a6b639a6ce service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:04:58.976415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22994a18-5101-4c8d-949f-b1a6b639a6ce service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:04:58.976696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22994a18-5101-4c8d-949f-b1a6b639a6ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:04:58.987243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22994a18-5101-4c8d-949f-b1a6b639a6ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:04:58.988685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22994a18-5101-4c8d-949f-b1a6b639a6ce service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:04:58.998067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1224/2477] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:04:58 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:00.150145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ac03f-7ed5-43bc-87c4-4754f7e8c21e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:00.157781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-870ac03f-7ed5-43bc-87c4-4754f7e8c21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:00.240185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ac03f-7ed5-43bc-87c4-4754f7e8c21e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:00.246196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f1f0b-7908-4d55-b2cd-bda4d36f3c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:00.246845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f1f0b-7908-4d55-b2cd-bda4d36f3c98 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:00.247125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f1f0b-7908-4d55-b2cd-bda4d36f3c98 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:00.247400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f1f0b-7908-4d55-b2cd-bda4d36f3c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:00.249178 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-245f1f0b-7908-4d55-b2cd-bda4d36f3c98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:00.262589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f1f0b-7908-4d55-b2cd-bda4d36f3c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:00.264243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f1f0b-7908-4d55-b2cd-bda4d36f3c98 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:00.275000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1254/2478] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:08.994407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:09.277509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22994a18-5101-4c8d-949f-b1a6b639a6ce service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:09.341657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22994a18-5101-4c8d-949f-b1a6b639a6ce service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:09.346934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:09.347796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:09.348077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:09.348350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] PATH_INFO: `/v3/users/f07c695b4137479da5998cae51bea565` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:09.349564 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f07c695b4137479da5998cae51bea565)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:09.351550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:09.507129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f07c695b4137479da5998cae51bea565'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:09.508002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f07c695b4137479da5998cae51bea565'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:09.520152 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f07c695b4137479da5998cae51bea565'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:09.526946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f07c695b4137479da5998cae51bea565'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:09.619546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f07c695b4137479da5998cae51bea565'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:09.626042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1225/2479] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:09 2022] DELETE /identity/v3/users/f07c695b4137479da5998cae51bea565 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:05:09.630931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c3f2e-0b73-449b-83df-786d5ed4d19f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:09.631433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c3f2e-0b73-449b-83df-786d5ed4d19f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:09.631687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c3f2e-0b73-449b-83df-786d5ed4d19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:09.651070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-656c3f2e-0b73-449b-83df-786d5ed4d19f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:09.652275 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-656c3f2e-0b73-449b-83df-786d5ed4d19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:09.695874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:09.718445 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1255/2480] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:05:09.728553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:09.730072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:09.787671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e748c6b-9623-48fa-9f05-6f3ca76ab8cf admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:09.791457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32982db-6543-4c13-b634-e1e26707b3d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:09.792108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32982db-6543-4c13-b634-e1e26707b3d2 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:09.792403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32982db-6543-4c13-b634-e1e26707b3d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:09.792684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32982db-6543-4c13-b634-e1e26707b3d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:09.804420 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32982db-6543-4c13-b634-e1e26707b3d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:09.805865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32982db-6543-4c13-b634-e1e26707b3d2 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:09.815547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1226/2481] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:10.278848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c3f2e-0b73-449b-83df-786d5ed4d19f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:10.280647 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-656c3f2e-0b73-449b-83df-786d5ed4d19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:10.294839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c3f2e-0b73-449b-83df-786d5ed4d19f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:10.301140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:10.302164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:10.302510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:10.302865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] PATH_INFO: `/v3/projects/d5fef7f41c2e413fb887cabe80e25e10` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:10.305229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5fef7f41c2e413fb887cabe80e25e10)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:10.307229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:10.396432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5fef7f41c2e413fb887cabe80e25e10', 'request_id': 'req-fd2c3977-8b30-484f-becf-ec54e4eca59f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:10.402769 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5fef7f41c2e413fb887cabe80e25e10', 'request_id': 'req-fd2c3977-8b30-484f-becf-ec54e4eca59f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:10.460073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5fef7f41c2e413fb887cabe80e25e10', 'request_id': 'req-fd2c3977-8b30-484f-becf-ec54e4eca59f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:10.474402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] The token cache is being invalidate because project d5fef7f41c2e413fb887cabe80e25e10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:10.474932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:10.475869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-fd2c3977-8b30-484f-becf-ec54e4eca59f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:10.477917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1256/2482] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:10 2022] DELETE /identity/v3/projects/d5fef7f41c2e413fb887cabe80e25e10 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:10.482918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bcf3b-d694-4d75-af90-755b5422a56a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:10.483425 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bcf3b-d694-4d75-af90-755b5422a56a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:10.483509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bcf3b-d694-4d75-af90-755b5422a56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:10.501540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-396bcf3b-d694-4d75-af90-755b5422a56a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:10.502839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-396bcf3b-d694-4d75-af90-755b5422a56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:10.532159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:10.579204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1227/2483] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:10.587545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca9b5-45f5-4e16-ae5a-831f191a8e56 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:10.587918 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca9b5-45f5-4e16-ae5a-831f191a8e56 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:10.588295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca9b5-45f5-4e16-ae5a-831f191a8e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:10.607949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-44bca9b5-45f5-4e16-ae5a-831f191a8e56 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:10.609361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-44bca9b5-45f5-4e16-ae5a-831f191a8e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:10.617617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1257/2484] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:10.621901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bcf3b-d694-4d75-af90-755b5422a56a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:10.631848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396bcf3b-d694-4d75-af90-755b5422a56a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:10.636984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e48c3-3742-4c22-90ab-78086a838459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:10.637641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e48c3-3742-4c22-90ab-78086a838459 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:10.637953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e48c3-3742-4c22-90ab-78086a838459 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:10.638236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e48c3-3742-4c22-90ab-78086a838459 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:10.638959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e48c3-3742-4c22-90ab-78086a838459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:05:10.639266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e48c3-3742-4c22-90ab-78086a838459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:10.641104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e48c3-3742-4c22-90ab-78086a838459 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:10.653972 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1228/2485] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:05:10.659043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bca9b5-45f5-4e16-ae5a-831f191a8e56 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:10.668537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bca9b5-45f5-4e16-ae5a-831f191a8e56 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:10.673173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a20edf-4020-4988-84f2-f28fd700a526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:10.673827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a20edf-4020-4988-84f2-f28fd700a526 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:10.674140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a20edf-4020-4988-84f2-f28fd700a526 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:10.674418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a20edf-4020-4988-84f2-f28fd700a526 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:10.675124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a20edf-4020-4988-84f2-f28fd700a526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:10.677076 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a20edf-4020-4988-84f2-f28fd700a526 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:10.770440 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1258/2486] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:10 2022] POST /identity/v3/projects => generated 401 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:10.774657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e48c3-3742-4c22-90ab-78086a838459 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:10.786233 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e48c3-3742-4c22-90ab-78086a838459 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:10.844337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b7202-4003-4adf-9a64-bb6c86956a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:10.844968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b7202-4003-4adf-9a64-bb6c86956a77 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:10.845255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b7202-4003-4adf-9a64-bb6c86956a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:10.845535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b7202-4003-4adf-9a64-bb6c86956a77 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:10.846308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b7202-4003-4adf-9a64-bb6c86956a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:10.848436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b7202-4003-4adf-9a64-bb6c86956a77 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:10.934726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1229/2487] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:10 2022] POST /identity/v3/users => generated 301 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:10.938686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a20edf-4020-4988-84f2-f28fd700a526 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:10.950131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a20edf-4020-4988-84f2-f28fd700a526 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:10.954923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe8b6d-3fc7-45fe-871e-a542d782c8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:10.956148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe8b6d-3fc7-45fe-871e-a542d782c8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:10.956433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe8b6d-3fc7-45fe-871e-a542d782c8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:10.956713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe8b6d-3fc7-45fe-871e-a542d782c8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:10.957444 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe8b6d-3fc7-45fe-871e-a542d782c8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:10.959064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe8b6d-3fc7-45fe-871e-a542d782c8e2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:10.969789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1259/2488] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:10.974073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b7202-4003-4adf-9a64-bb6c86956a77 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:10.983962 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b7202-4003-4adf-9a64-bb6c86956a77 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:10.988958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7561c-fb2e-480f-ad4c-32ef8c5c2f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:10.990254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7561c-fb2e-480f-ad4c-32ef8c5c2f74 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:10.990760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7561c-fb2e-480f-ad4c-32ef8c5c2f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:10.991040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7561c-fb2e-480f-ad4c-32ef8c5c2f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:10.991838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7561c-fb2e-480f-ad4c-32ef8c5c2f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:10.993678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7561c-fb2e-480f-ad4c-32ef8c5c2f74 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:11.008555 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1230/2489] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:11.009990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe8b6d-3fc7-45fe-871e-a542d782c8e2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:11.021303 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe8b6d-3fc7-45fe-871e-a542d782c8e2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:11.026177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554e8a1-0170-4189-b4d3-9bfaf1a1ee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:11.027058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554e8a1-0170-4189-b4d3-9bfaf1a1ee9a admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:11.027344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554e8a1-0170-4189-b4d3-9bfaf1a1ee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:11.027621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554e8a1-0170-4189-b4d3-9bfaf1a1ee9a admin admin] PATH_INFO: `/v3/projects/60b839484e3946899a2a4501b420156f/users/71bbf8e256b64777979b03983fb96a39/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:11.043066 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554e8a1-0170-4189-b4d3-9bfaf1a1ee9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b839484e3946899a2a4501b420156f, user_id=71bbf8e256b64777979b03983fb96a39, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:11.045720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554e8a1-0170-4189-b4d3-9bfaf1a1ee9a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:11.117650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1260/2490] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:11 2022] PUT /identity/v3/projects/60b839484e3946899a2a4501b420156f/users/71bbf8e256b64777979b03983fb96a39/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:11.129402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082347b-497a-41da-9893-f76566969635 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:11.129719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082347b-497a-41da-9893-f76566969635 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:11.130017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082347b-497a-41da-9893-f76566969635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:11.159442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f082347b-497a-41da-9893-f76566969635 None None] MFA Rules not processed for user `71bbf8e256b64777979b03983fb96a39`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:11.160698 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f082347b-497a-41da-9893-f76566969635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:11.244438 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1231/2491] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:05:11.248799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554e8a1-0170-4189-b4d3-9bfaf1a1ee9a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:11.321190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554e8a1-0170-4189-b4d3-9bfaf1a1ee9a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:11.328350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27c02d-8d01-463b-a4a2-71f7cfea79cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:11.329205 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27c02d-8d01-463b-a4a2-71f7cfea79cc admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:11.329491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27c02d-8d01-463b-a4a2-71f7cfea79cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:11.330622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27c02d-8d01-463b-a4a2-71f7cfea79cc admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:11.331330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27c02d-8d01-463b-a4a2-71f7cfea79cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:11.335388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27c02d-8d01-463b-a4a2-71f7cfea79cc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:11.346188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:11.438989 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1261/2492] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:11 2022] POST /identity/v3/projects => generated 405 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:11.444332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082347b-497a-41da-9893-f76566969635 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:11.456667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082347b-497a-41da-9893-f76566969635 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:11.470684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:11.524123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed84d7d-812e-403e-b453-687260b166bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:11.524967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed84d7d-812e-403e-b453-687260b166bd admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:11.525087 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed84d7d-812e-403e-b453-687260b166bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:11.525546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed84d7d-812e-403e-b453-687260b166bd admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:11.526610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed84d7d-812e-403e-b453-687260b166bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:11.528700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed84d7d-812e-403e-b453-687260b166bd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:11.632347 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1232/2493] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:11 2022] POST /identity/v3/users => generated 309 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:11.636694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27c02d-8d01-463b-a4a2-71f7cfea79cc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:11.649516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27c02d-8d01-463b-a4a2-71f7cfea79cc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:11.656190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc279a-1db3-488b-8d94-fc96b929de15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:11.656888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc279a-1db3-488b-8d94-fc96b929de15 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:11.657618 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc279a-1db3-488b-8d94-fc96b929de15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:11.657737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc279a-1db3-488b-8d94-fc96b929de15 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:11.658968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc279a-1db3-488b-8d94-fc96b929de15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:11.662980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc279a-1db3-488b-8d94-fc96b929de15 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:11.673254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1262/2494] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:11.676711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed84d7d-812e-403e-b453-687260b166bd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:11.689845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed84d7d-812e-403e-b453-687260b166bd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:11.693913 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2d662-4566-4099-9f3c-35b29ec17913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:11.695589 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2d662-4566-4099-9f3c-35b29ec17913 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:11.695883 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2d662-4566-4099-9f3c-35b29ec17913 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:11.696637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2d662-4566-4099-9f3c-35b29ec17913 admin admin] PATH_INFO: `/v3/projects/10badd6603e940f39725077ef42232e1/users/bd4b619fe581494783055783e8b73b62/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:11.714540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2d662-4566-4099-9f3c-35b29ec17913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10badd6603e940f39725077ef42232e1, user_id=bd4b619fe581494783055783e8b73b62, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:11.717408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2d662-4566-4099-9f3c-35b29ec17913 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:11.815115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1233/2495] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:11 2022] PUT /identity/v3/projects/10badd6603e940f39725077ef42232e1/users/bd4b619fe581494783055783e8b73b62/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:11.818916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc279a-1db3-488b-8d94-fc96b929de15 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:11.829565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc279a-1db3-488b-8d94-fc96b929de15 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:11.893100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9b622-c6dd-4ffa-8a72-ddea164439d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:11.893793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9b622-c6dd-4ffa-8a72-ddea164439d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:11.894121 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9b622-c6dd-4ffa-8a72-ddea164439d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:11.894406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9b622-c6dd-4ffa-8a72-ddea164439d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:11.895945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9b622-c6dd-4ffa-8a72-ddea164439d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:11.896822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9b622-c6dd-4ffa-8a72-ddea164439d9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:11.909641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1263/2496] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:11.913756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2d662-4566-4099-9f3c-35b29ec17913 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:11.925389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2d662-4566-4099-9f3c-35b29ec17913 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:11.930261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d01c0-0ebd-42aa-b24f-696a333a6c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:11.931148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d01c0-0ebd-42aa-b24f-696a333a6c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:11.931431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d01c0-0ebd-42aa-b24f-696a333a6c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:11.931710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d01c0-0ebd-42aa-b24f-696a333a6c07 admin admin] PATH_INFO: `/v3/projects/10badd6603e940f39725077ef42232e1/users/bd4b619fe581494783055783e8b73b62/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:11.933653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d01c0-0ebd-42aa-b24f-696a333a6c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10badd6603e940f39725077ef42232e1, user_id=bd4b619fe581494783055783e8b73b62, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:11.936331 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d01c0-0ebd-42aa-b24f-696a333a6c07 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:11.948802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1234/2497] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:11 2022] PUT /identity/v3/projects/10badd6603e940f39725077ef42232e1/users/bd4b619fe581494783055783e8b73b62/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:11.959725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5b459-e742-4a03-bf79-2e5cb2bb0b70 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:11.960012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5b459-e742-4a03-bf79-2e5cb2bb0b70 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:11.960291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5b459-e742-4a03-bf79-2e5cb2bb0b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:11.991514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c8d5b459-e742-4a03-bf79-2e5cb2bb0b70 None None] MFA Rules not processed for user `bd4b619fe581494783055783e8b73b62`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:11.992814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c8d5b459-e742-4a03-bf79-2e5cb2bb0b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:12.062517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1264/2498] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:12.073652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d01c0-0ebd-42aa-b24f-696a333a6c07 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:12.075548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fb5d01c0-0ebd-42aa-b24f-696a333a6c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:12.141394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d01c0-0ebd-42aa-b24f-696a333a6c07 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:12.145726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c1dc2-4140-4aff-b69c-282102bad7c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:12.146483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c1dc2-4140-4aff-b69c-282102bad7c6 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:12.146837 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c1dc2-4140-4aff-b69c-282102bad7c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:12.147184 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c1dc2-4140-4aff-b69c-282102bad7c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:12.157542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c1dc2-4140-4aff-b69c-282102bad7c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:12.159584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c1dc2-4140-4aff-b69c-282102bad7c6 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:12.169920 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1235/2499] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:12 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:12.304295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5b459-e742-4a03-bf79-2e5cb2bb0b70 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:12.368221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5b459-e742-4a03-bf79-2e5cb2bb0b70 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:12.372762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:12.373629 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:12.373933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:12.374214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] PATH_INFO: `/v3/users/71bbf8e256b64777979b03983fb96a39` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:12.375405 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bbf8e256b64777979b03983fb96a39)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:12.377868 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:12.380447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:12.552924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bbf8e256b64777979b03983fb96a39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:12.553579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bbf8e256b64777979b03983fb96a39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:12.560470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bbf8e256b64777979b03983fb96a39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:12.573579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bbf8e256b64777979b03983fb96a39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:12.636585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bbf8e256b64777979b03983fb96a39'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:12.643476 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1265/2500] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:12 2022] DELETE /identity/v3/users/71bbf8e256b64777979b03983fb96a39 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:12.648189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb9da-69ad-4dda-b301-b49db6a68ec5 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:12.648675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb9da-69ad-4dda-b301-b49db6a68ec5 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:12.648793 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb9da-69ad-4dda-b301-b49db6a68ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:12.666212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e1bbb9da-69ad-4dda-b301-b49db6a68ec5 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:12.667407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e1bbb9da-69ad-4dda-b301-b49db6a68ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:12.727688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1236/2501] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:12.737817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:12.739266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:12.795173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650306e0-ab97-4a8e-8cac-6842879d65d3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:12.799360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8030166-bc1b-4867-8775-1e093eedd77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:12.799993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8030166-bc1b-4867-8775-1e093eedd77c service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:12.800277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8030166-bc1b-4867-8775-1e093eedd77c service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:12.800545 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8030166-bc1b-4867-8775-1e093eedd77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:12.810059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8030166-bc1b-4867-8775-1e093eedd77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:12.811815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8030166-bc1b-4867-8775-1e093eedd77c service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:12.821822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1266/2502] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:13.261311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb9da-69ad-4dda-b301-b49db6a68ec5 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:13.263384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e1bbb9da-69ad-4dda-b301-b49db6a68ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:13.276361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb9da-69ad-4dda-b301-b49db6a68ec5 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:13.281332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:13.282256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:13.282543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:13.283798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] PATH_INFO: `/v3/projects/60b839484e3946899a2a4501b420156f` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:13.284672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b839484e3946899a2a4501b420156f)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:13.286661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:13.295785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:13.397406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b839484e3946899a2a4501b420156f', 'request_id': 'req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.402935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b839484e3946899a2a4501b420156f', 'request_id': 'req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.482956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b839484e3946899a2a4501b420156f', 'request_id': 'req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.495527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] The token cache is being invalidate because project 60b839484e3946899a2a4501b420156f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:13.495810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.496463 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.498072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1237/2503] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:13 2022] DELETE /identity/v3/projects/60b839484e3946899a2a4501b420156f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:13.502042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8030166-bc1b-4867-8775-1e093eedd77c service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:13.512009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8030166-bc1b-4867-8775-1e093eedd77c service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:13.569166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:13.570100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:13.570388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:13.570695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] PATH_INFO: `/v3/users/bd4b619fe581494783055783e8b73b62` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:13.571900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd4b619fe581494783055783e8b73b62)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:13.573904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:13.671610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:13.742027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd4b619fe581494783055783e8b73b62'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.742686 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd4b619fe581494783055783e8b73b62'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.749646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd4b619fe581494783055783e8b73b62'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.761584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd4b619fe581494783055783e8b73b62'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.807577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-65fa6485-259f-40a7-8348-c2862dff718d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd4b619fe581494783055783e8b73b62'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:13.813801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1267/2504] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:13 2022] DELETE /identity/v3/users/bd4b619fe581494783055783e8b73b62 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:14.307468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:14.308997 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:14.324427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7b05e-17e8-4daf-9c7f-2b70798f6a2d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:14.381132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:14.382011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:14.382291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:14.382832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] PATH_INFO: `/v3/projects/10badd6603e940f39725077ef42232e1` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:14.383984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10badd6603e940f39725077ef42232e1)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:14.385907 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:14.482537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10badd6603e940f39725077ef42232e1', 'request_id': 'req-86a53a82-d50b-469c-9b8b-de86095cac91'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:14.488401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10badd6603e940f39725077ef42232e1', 'request_id': 'req-86a53a82-d50b-469c-9b8b-de86095cac91'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:14.544911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10badd6603e940f39725077ef42232e1', 'request_id': 'req-86a53a82-d50b-469c-9b8b-de86095cac91'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:14.557115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] The token cache is being invalidate because project 10badd6603e940f39725077ef42232e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:14.557403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:14.558061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-86a53a82-d50b-469c-9b8b-de86095cac91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:14.559614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1238/2505] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:14 2022] DELETE /identity/v3/projects/10badd6603e940f39725077ef42232e1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:14.565607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd39e97-e4d2-4fa4-b8e1-b4d47ff2a13a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:14.565607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd39e97-e4d2-4fa4-b8e1-b4d47ff2a13a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:14.565607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd39e97-e4d2-4fa4-b8e1-b4d47ff2a13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:14.583267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6dd39e97-e4d2-4fa4-b8e1-b4d47ff2a13a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:14.584456 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6dd39e97-e4d2-4fa4-b8e1-b4d47ff2a13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:14.643167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1268/2506] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:14.652159 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb386-b1bf-4e25-8ac1-757c64ccd9ee None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:14.652455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb386-b1bf-4e25-8ac1-757c64ccd9ee None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:14.652767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb386-b1bf-4e25-8ac1-757c64ccd9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:14.671482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2dabb386-b1bf-4e25-8ac1-757c64ccd9ee None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:14.672689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2dabb386-b1bf-4e25-8ac1-757c64ccd9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:14.681199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1239/2507] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:14.685359 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd39e97-e4d2-4fa4-b8e1-b4d47ff2a13a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:14.694796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd39e97-e4d2-4fa4-b8e1-b4d47ff2a13a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:14.699330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56987d-a806-4def-a838-0da1fcfa8f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:14.699976 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56987d-a806-4def-a838-0da1fcfa8f47 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:14.700259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56987d-a806-4def-a838-0da1fcfa8f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:14.700538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56987d-a806-4def-a838-0da1fcfa8f47 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:14.701260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56987d-a806-4def-a838-0da1fcfa8f47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:05:14.701567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56987d-a806-4def-a838-0da1fcfa8f47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:14.703192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56987d-a806-4def-a838-0da1fcfa8f47 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:14.714495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1269/2508] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:05:14.718893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabb386-b1bf-4e25-8ac1-757c64ccd9ee None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:14.728619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabb386-b1bf-4e25-8ac1-757c64ccd9ee None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:14.733201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ec7b7-7fa9-462e-9065-1f8fbde6b236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:14.733791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ec7b7-7fa9-462e-9065-1f8fbde6b236 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:14.734107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ec7b7-7fa9-462e-9065-1f8fbde6b236 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:14.734436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ec7b7-7fa9-462e-9065-1f8fbde6b236 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:14.735112 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ec7b7-7fa9-462e-9065-1f8fbde6b236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:14.737643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ec7b7-7fa9-462e-9065-1f8fbde6b236 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:14.819807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:14.826806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1240/2509] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:14 2022] POST /identity/v3/projects => generated 361 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:14.830872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56987d-a806-4def-a838-0da1fcfa8f47 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:14.840944 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56987d-a806-4def-a838-0da1fcfa8f47 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:14.896983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42752-982c-40ed-909d-be594c2515da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:14.897648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42752-982c-40ed-909d-be594c2515da admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:14.897967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42752-982c-40ed-909d-be594c2515da admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:14.898253 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c42752-982c-40ed-909d-be594c2515da admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:14.898954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42752-982c-40ed-909d-be594c2515da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:14.900872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c42752-982c-40ed-909d-be594c2515da admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:14.979277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1270/2510] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:14 2022] POST /identity/v3/users => generated 281 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:14.983630 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ec7b7-7fa9-462e-9065-1f8fbde6b236 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:14.994248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ec7b7-7fa9-462e-9065-1f8fbde6b236 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:14.999002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121d5ff-8034-456b-a374-40972fd825e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:14.999685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121d5ff-8034-456b-a374-40972fd825e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:14.999972 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121d5ff-8034-456b-a374-40972fd825e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.000252 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121d5ff-8034-456b-a374-40972fd825e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.000976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121d5ff-8034-456b-a374-40972fd825e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:15.002627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121d5ff-8034-456b-a374-40972fd825e2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:15.013133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1241/2511] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:05:15.017010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42752-982c-40ed-909d-be594c2515da admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:15.027299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c42752-982c-40ed-909d-be594c2515da admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:15.031792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d2adc-e0c7-4f62-be3f-9fbf7540795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:15.032436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d2adc-e0c7-4f62-be3f-9fbf7540795e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.032719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d2adc-e0c7-4f62-be3f-9fbf7540795e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.032995 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d2adc-e0c7-4f62-be3f-9fbf7540795e admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.034443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d2adc-e0c7-4f62-be3f-9fbf7540795e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:15.036026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d2adc-e0c7-4f62-be3f-9fbf7540795e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:15.045679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1271/2512] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:15.049603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121d5ff-8034-456b-a374-40972fd825e2 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:15.059281 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121d5ff-8034-456b-a374-40972fd825e2 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:15.063786 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e1303-c345-4eb0-98fd-d02abe177d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:15.064680 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e1303-c345-4eb0-98fd-d02abe177d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.064963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e1303-c345-4eb0-98fd-d02abe177d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.065240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e1303-c345-4eb0-98fd-d02abe177d82 admin admin] PATH_INFO: `/v3/projects/4fe14a4de5e14a9789fc2d24dd79b043/users/08c43417c571459e90895c1da1344a8d/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.080938 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e1303-c345-4eb0-98fd-d02abe177d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe14a4de5e14a9789fc2d24dd79b043, user_id=08c43417c571459e90895c1da1344a8d, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:15.083624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e1303-c345-4eb0-98fd-d02abe177d82 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:15.139565 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1242/2513] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:15 2022] PUT /identity/v3/projects/4fe14a4de5e14a9789fc2d24dd79b043/users/08c43417c571459e90895c1da1344a8d/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:05:15.150588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040d72-4077-4b4b-bef9-e3e7c3c120a0 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.150917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040d72-4077-4b4b-bef9-e3e7c3c120a0 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.151201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4040d72-4077-4b4b-bef9-e3e7c3c120a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.170031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:15.178371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d4040d72-4077-4b4b-bef9-e3e7c3c120a0 None None] MFA Rules not processed for user `08c43417c571459e90895c1da1344a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:15.179571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d4040d72-4077-4b4b-bef9-e3e7c3c120a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:15.245236 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1272/2514] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:15.249720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e1303-c345-4eb0-98fd-d02abe177d82 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:15.318865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e1303-c345-4eb0-98fd-d02abe177d82 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:15.323830 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a1b75-bf82-429f-91fa-14e158396ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:15.324497 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a1b75-bf82-429f-91fa-14e158396ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.324780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a1b75-bf82-429f-91fa-14e158396ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.325054 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a1b75-bf82-429f-91fa-14e158396ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.325743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a1b75-bf82-429f-91fa-14e158396ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:15.328190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a1b75-bf82-429f-91fa-14e158396ef4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:15.418481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1243/2515] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:15 2022] POST /identity/v3/projects => generated 363 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:15.419709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040d72-4077-4b4b-bef9-e3e7c3c120a0 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:15.430553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4040d72-4077-4b4b-bef9-e3e7c3c120a0 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:15.488492 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011c6c4-e6b0-4ecd-a2ee-8ac5e616c616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:15.489236 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011c6c4-e6b0-4ecd-a2ee-8ac5e616c616 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.498546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011c6c4-e6b0-4ecd-a2ee-8ac5e616c616 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.498546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e011c6c4-e6b0-4ecd-a2ee-8ac5e616c616 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.498546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011c6c4-e6b0-4ecd-a2ee-8ac5e616c616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:15.498546 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e011c6c4-e6b0-4ecd-a2ee-8ac5e616c616 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:15.586613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1273/2516] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:15 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:15.590735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a1b75-bf82-429f-91fa-14e158396ef4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:15.601276 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a1b75-bf82-429f-91fa-14e158396ef4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:15.605922 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151de8-539e-4593-a841-7ad8d986f3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:15.606588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151de8-539e-4593-a841-7ad8d986f3eb admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.606891 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151de8-539e-4593-a841-7ad8d986f3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.607173 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151de8-539e-4593-a841-7ad8d986f3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.607887 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a151de8-539e-4593-a841-7ad8d986f3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:15.609481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a151de8-539e-4593-a841-7ad8d986f3eb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:15.620096 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1244/2517] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:15.624319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011c6c4-e6b0-4ecd-a2ee-8ac5e616c616 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:15.635279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e011c6c4-e6b0-4ecd-a2ee-8ac5e616c616 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:15.640225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fa806-660b-4ae0-bfd9-55b7dfa528ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:15.641107 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fa806-660b-4ae0-bfd9-55b7dfa528ef admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.641416 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fa806-660b-4ae0-bfd9-55b7dfa528ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.641702 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fa806-660b-4ae0-bfd9-55b7dfa528ef admin admin] PATH_INFO: `/v3/projects/fdd370ccbf5b4980a195e8c75e4cb073/users/6ccf95ad21d44845a8cba0135f06add4/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.657288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fa806-660b-4ae0-bfd9-55b7dfa528ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd370ccbf5b4980a195e8c75e4cb073, user_id=6ccf95ad21d44845a8cba0135f06add4, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:15.660142 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fa806-660b-4ae0-bfd9-55b7dfa528ef admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:15.752951 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1274/2518] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:15 2022] PUT /identity/v3/projects/fdd370ccbf5b4980a195e8c75e4cb073/users/6ccf95ad21d44845a8cba0135f06add4/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:15.756949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151de8-539e-4593-a841-7ad8d986f3eb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:15.767995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151de8-539e-4593-a841-7ad8d986f3eb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:15.824976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87c344-e7a1-40b3-8312-e685e2a2eef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:15.825647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c344-e7a1-40b3-8312-e685e2a2eef4 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.825974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c344-e7a1-40b3-8312-e685e2a2eef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.833011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87c344-e7a1-40b3-8312-e685e2a2eef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.833011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87c344-e7a1-40b3-8312-e685e2a2eef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:15.833011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87c344-e7a1-40b3-8312-e685e2a2eef4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:15.839339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1245/2519] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:15.843412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fa806-660b-4ae0-bfd9-55b7dfa528ef admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:15.854059 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fa806-660b-4ae0-bfd9-55b7dfa528ef admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:15.858700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2e44d-9d28-437a-9510-fcea61fe7852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:15.860755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2e44d-9d28-437a-9510-fcea61fe7852 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.860755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2e44d-9d28-437a-9510-fcea61fe7852 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.860755 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2e44d-9d28-437a-9510-fcea61fe7852 admin admin] PATH_INFO: `/v3/projects/fdd370ccbf5b4980a195e8c75e4cb073/users/6ccf95ad21d44845a8cba0135f06add4/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.862740 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2e44d-9d28-437a-9510-fcea61fe7852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd370ccbf5b4980a195e8c75e4cb073, user_id=6ccf95ad21d44845a8cba0135f06add4, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:15.865371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2e44d-9d28-437a-9510-fcea61fe7852 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:15.875637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1275/2520] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:15 2022] PUT /identity/v3/projects/fdd370ccbf5b4980a195e8c75e4cb073/users/6ccf95ad21d44845a8cba0135f06add4/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:15.886530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c9a8a-b402-421e-9a56-971cfe66e147 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:15.889494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c9a8a-b402-421e-9a56-971cfe66e147 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:15.889494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c9a8a-b402-421e-9a56-971cfe66e147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:15.913256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e98c9a8a-b402-421e-9a56-971cfe66e147 None None] MFA Rules not processed for user `6ccf95ad21d44845a8cba0135f06add4`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:15.914485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e98c9a8a-b402-421e-9a56-971cfe66e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:15.934663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:15.981855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1246/2521] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:15.993573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2e44d-9d28-437a-9510-fcea61fe7852 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:15.995311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-18c2e44d-9d28-437a-9510-fcea61fe7852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:16.053799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2e44d-9d28-437a-9510-fcea61fe7852 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:16.057437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412d275-1391-4e59-b281-e0519882c79b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:16.058109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412d275-1391-4e59-b281-e0519882c79b service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:16.058399 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412d275-1391-4e59-b281-e0519882c79b service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:16.058677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412d275-1391-4e59-b281-e0519882c79b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:16.064652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:16.068154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412d275-1391-4e59-b281-e0519882c79b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:16.069581 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412d275-1391-4e59-b281-e0519882c79b service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:16.079177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1276/2522] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:17.024825 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c9a8a-b402-421e-9a56-971cfe66e147 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:17.088382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c9a8a-b402-421e-9a56-971cfe66e147 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:17.092912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:17.093775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:17.094086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:17.094363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] PATH_INFO: `/v3/users/08c43417c571459e90895c1da1344a8d` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:17.095526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c43417c571459e90895c1da1344a8d)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:17.097964 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:17.342724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c43417c571459e90895c1da1344a8d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:17.343648 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c43417c571459e90895c1da1344a8d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:17.356626 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c43417c571459e90895c1da1344a8d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:17.363027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c43417c571459e90895c1da1344a8d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:17.448002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c43417c571459e90895c1da1344a8d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:17.454150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1247/2523] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:17 2022] DELETE /identity/v3/users/08c43417c571459e90895c1da1344a8d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:17.459143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabc4e-0a89-44c1-b345-c4d0d6e39770 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:17.459632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabc4e-0a89-44c1-b345-c4d0d6e39770 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:17.459737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabc4e-0a89-44c1-b345-c4d0d6e39770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:17.476848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0daabc4e-0a89-44c1-b345-c4d0d6e39770 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:17.478091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0daabc4e-0a89-44c1-b345-c4d0d6e39770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:17.536677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1277/2524] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:17.546484 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:17.548007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:17.592608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:17.611065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc4204-7103-44ed-8e3f-bb8f6bd1937f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:17.614983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1dc82-a36b-4dc7-9e26-f2acdb859e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:17.615628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1dc82-a36b-4dc7-9e26-f2acdb859e5e service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:17.615912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1dc82-a36b-4dc7-9e26-f2acdb859e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:17.616190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1dc82-a36b-4dc7-9e26-f2acdb859e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:17.625308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1dc82-a36b-4dc7-9e26-f2acdb859e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:17.626769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1dc82-a36b-4dc7-9e26-f2acdb859e5e service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:17.637080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1248/2525] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:18.123876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daabc4e-0a89-44c1-b345-c4d0d6e39770 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:18.125596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0daabc4e-0a89-44c1-b345-c4d0d6e39770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:18.139003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daabc4e-0a89-44c1-b345-c4d0d6e39770 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:18.143896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:18.144829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:18.145130 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:18.145458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] PATH_INFO: `/v3/projects/4fe14a4de5e14a9789fc2d24dd79b043` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:18.146938 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe14a4de5e14a9789fc2d24dd79b043)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:18.149219 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:18.301432 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe14a4de5e14a9789fc2d24dd79b043', 'request_id': 'req-ede7ae01-4345-4a21-910a-87e8d9e8902c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.307709 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe14a4de5e14a9789fc2d24dd79b043', 'request_id': 'req-ede7ae01-4345-4a21-910a-87e8d9e8902c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.374159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe14a4de5e14a9789fc2d24dd79b043', 'request_id': 'req-ede7ae01-4345-4a21-910a-87e8d9e8902c'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.386369 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] The token cache is being invalidate because project 4fe14a4de5e14a9789fc2d24dd79b043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:18.386664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.387435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.389077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1278/2526] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:18 2022] DELETE /identity/v3/projects/4fe14a4de5e14a9789fc2d24dd79b043 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:18.393061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1dc82-a36b-4dc7-9e26-f2acdb859e5e service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:18.403067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1dc82-a36b-4dc7-9e26-f2acdb859e5e service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:18.460473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:18.461356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:18.461672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:18.461984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] PATH_INFO: `/v3/users/6ccf95ad21d44845a8cba0135f06add4` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:18.463176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ccf95ad21d44845a8cba0135f06add4)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:18.465135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:18.673939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ccf95ad21d44845a8cba0135f06add4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.674649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ccf95ad21d44845a8cba0135f06add4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.686757 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ccf95ad21d44845a8cba0135f06add4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.693382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ccf95ad21d44845a8cba0135f06add4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.773877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-82b017d5-e307-4b9d-addb-f6e368c6a2d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ccf95ad21d44845a8cba0135f06add4'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:18.780376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1249/2527] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:18 2022] DELETE /identity/v3/users/6ccf95ad21d44845a8cba0135f06add4 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:19.358251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:19.359767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:19.373334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7ae01-4345-4a21-910a-87e8d9e8902c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:19.433921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:19.434835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:19.435120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:19.435429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] PATH_INFO: `/v3/projects/fdd370ccbf5b4980a195e8c75e4cb073` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:19.436703 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd370ccbf5b4980a195e8c75e4cb073)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:19.438756 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:19.461327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:19.551535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd370ccbf5b4980a195e8c75e4cb073', 'request_id': 'req-67655d2c-1b5b-458e-afc7-c9be51d798a0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:19.558634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd370ccbf5b4980a195e8c75e4cb073', 'request_id': 'req-67655d2c-1b5b-458e-afc7-c9be51d798a0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:19.653436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd370ccbf5b4980a195e8c75e4cb073', 'request_id': 'req-67655d2c-1b5b-458e-afc7-c9be51d798a0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:19.665812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] The token cache is being invalidate because project fdd370ccbf5b4980a195e8c75e4cb073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:19.666143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:19.666813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-67655d2c-1b5b-458e-afc7-c9be51d798a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:19.668441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1279/2528] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:19 2022] DELETE /identity/v3/projects/fdd370ccbf5b4980a195e8c75e4cb073 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:19.673422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf77a9-0067-40e0-a33c-05758d372523 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:19.673946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf77a9-0067-40e0-a33c-05758d372523 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:19.674098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf77a9-0067-40e0-a33c-05758d372523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:19.693437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-c4cf77a9-0067-40e0-a33c-05758d372523 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:19.694661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c4cf77a9-0067-40e0-a33c-05758d372523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:19.753999 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:19.762356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1250/2529] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:05:19.770950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27501cc-0f1b-4c91-abc5-5950a2d3be42 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:19.771326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27501cc-0f1b-4c91-abc5-5950a2d3be42 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:19.771676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27501cc-0f1b-4c91-abc5-5950a2d3be42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:19.791117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-b27501cc-0f1b-4c91-abc5-5950a2d3be42 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:19.792633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b27501cc-0f1b-4c91-abc5-5950a2d3be42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:19.801804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1280/2530] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:19.806319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf77a9-0067-40e0-a33c-05758d372523 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:19.816153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf77a9-0067-40e0-a33c-05758d372523 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:19.821459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeadc6c-e4c1-4561-a6b6-21650b0aaa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:19.822143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeadc6c-e4c1-4561-a6b6-21650b0aaa99 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:19.822427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeadc6c-e4c1-4561-a6b6-21650b0aaa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:19.822703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeadc6c-e4c1-4561-a6b6-21650b0aaa99 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:19.823573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeadc6c-e4c1-4561-a6b6-21650b0aaa99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:05:19.823889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeadc6c-e4c1-4561-a6b6-21650b0aaa99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:19.825519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeadc6c-e4c1-4561-a6b6-21650b0aaa99 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:19.837669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1251/2531] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:05:19.842095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27501cc-0f1b-4c91-abc5-5950a2d3be42 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:19.851910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27501cc-0f1b-4c91-abc5-5950a2d3be42 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:19.858999 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5170624-9292-4312-94a2-3c48677a2646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:19.859974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5170624-9292-4312-94a2-3c48677a2646 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:19.859974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5170624-9292-4312-94a2-3c48677a2646 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:19.859974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5170624-9292-4312-94a2-3c48677a2646 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:19.860521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5170624-9292-4312-94a2-3c48677a2646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:19.862446 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5170624-9292-4312-94a2-3c48677a2646 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:19.947063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1281/2532] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:19 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:19.951514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeadc6c-e4c1-4561-a6b6-21650b0aaa99 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:19.961850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeadc6c-e4c1-4561-a6b6-21650b0aaa99 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:20.017494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147c44-7fa7-4800-9599-67880fe4ee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:20.018229 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147c44-7fa7-4800-9599-67880fe4ee3f admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:20.018519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147c44-7fa7-4800-9599-67880fe4ee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:20.018798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce147c44-7fa7-4800-9599-67880fe4ee3f admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:20.019525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce147c44-7fa7-4800-9599-67880fe4ee3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:20.021442 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce147c44-7fa7-4800-9599-67880fe4ee3f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:20.111174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1252/2533] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:19 2022] POST /identity/v3/users => generated 282 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:20.116475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5170624-9292-4312-94a2-3c48677a2646 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:20.134211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5170624-9292-4312-94a2-3c48677a2646 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:20.139426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28acc6-e125-43ba-8532-e8609f641b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:20.140233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28acc6-e125-43ba-8532-e8609f641b6c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:20.140617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28acc6-e125-43ba-8532-e8609f641b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:20.140990 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28acc6-e125-43ba-8532-e8609f641b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:20.141852 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28acc6-e125-43ba-8532-e8609f641b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:20.155406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28acc6-e125-43ba-8532-e8609f641b6c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:20.157339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1282/2534] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:20.162269 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147c44-7fa7-4800-9599-67880fe4ee3f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:20.173025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce147c44-7fa7-4800-9599-67880fe4ee3f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:20.178334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4166ebe-73c9-410c-92d7-fc13a45a7666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:20.178479 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4166ebe-73c9-410c-92d7-fc13a45a7666 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:20.178765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4166ebe-73c9-410c-92d7-fc13a45a7666 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:20.179043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4166ebe-73c9-410c-92d7-fc13a45a7666 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:20.179761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4166ebe-73c9-410c-92d7-fc13a45a7666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:20.181387 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4166ebe-73c9-410c-92d7-fc13a45a7666 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:20.191739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1253/2535] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:20.196085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28acc6-e125-43ba-8532-e8609f641b6c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:20.206626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28acc6-e125-43ba-8532-e8609f641b6c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:20.211818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a617b0e-e22a-4bc7-9ce3-6fcec723ab4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:20.212731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a617b0e-e22a-4bc7-9ce3-6fcec723ab4c admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:20.213015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a617b0e-e22a-4bc7-9ce3-6fcec723ab4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:20.213409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a617b0e-e22a-4bc7-9ce3-6fcec723ab4c admin admin] PATH_INFO: `/v3/projects/6df38c9fa8264a229fc71c49bd9143ab/users/f6c65710fce44d12a8d559431ac866a8/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:20.229593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a617b0e-e22a-4bc7-9ce3-6fcec723ab4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df38c9fa8264a229fc71c49bd9143ab, user_id=f6c65710fce44d12a8d559431ac866a8, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:20.232324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a617b0e-e22a-4bc7-9ce3-6fcec723ab4c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:20.305614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1283/2536] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:20 2022] PUT /identity/v3/projects/6df38c9fa8264a229fc71c49bd9143ab/users/f6c65710fce44d12a8d559431ac866a8/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:20.316773 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376078f-0f89-4809-85f8-a9d9636c6b08 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:20.317616 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376078f-0f89-4809-85f8-a9d9636c6b08 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:20.317936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376078f-0f89-4809-85f8-a9d9636c6b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:20.345000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8376078f-0f89-4809-85f8-a9d9636c6b08 None None] MFA Rules not processed for user `f6c65710fce44d12a8d559431ac866a8`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:20.346226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8376078f-0f89-4809-85f8-a9d9636c6b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:20.410239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1254/2537] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:20.415192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a617b0e-e22a-4bc7-9ce3-6fcec723ab4c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:20.481234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a617b0e-e22a-4bc7-9ce3-6fcec723ab4c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:20.486038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c50a5a-a0b6-4055-b476-e04031790fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:20.486718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c50a5a-a0b6-4055-b476-e04031790fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:20.487061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c50a5a-a0b6-4055-b476-e04031790fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:20.487341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c50a5a-a0b6-4055-b476-e04031790fa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:20.488519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c50a5a-a0b6-4055-b476-e04031790fa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:20.490477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c50a5a-a0b6-4055-b476-e04031790fa7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:20.585584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1284/2538] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:20 2022] POST /identity/v3/projects => generated 363 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:20.587350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376078f-0f89-4809-85f8-a9d9636c6b08 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:20.603550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376078f-0f89-4809-85f8-a9d9636c6b08 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:20.684017 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:20.691258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb2eaf-830a-4fb1-8541-46ad44391e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:20.692596 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb2eaf-830a-4fb1-8541-46ad44391e94 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:20.692885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb2eaf-830a-4fb1-8541-46ad44391e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:20.693162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb2eaf-830a-4fb1-8541-46ad44391e94 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:20.693862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb2eaf-830a-4fb1-8541-46ad44391e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:20.696013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb2eaf-830a-4fb1-8541-46ad44391e94 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:20.799026 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1255/2539] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:20 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:20.804617 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c50a5a-a0b6-4055-b476-e04031790fa7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:20.822226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c50a5a-a0b6-4055-b476-e04031790fa7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:20.836372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e554d78-7e98-419a-bf41-74b4bc7f6052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:20.836372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554d78-7e98-419a-bf41-74b4bc7f6052 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:20.836372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554d78-7e98-419a-bf41-74b4bc7f6052 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:20.836372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554d78-7e98-419a-bf41-74b4bc7f6052 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:20.836876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e554d78-7e98-419a-bf41-74b4bc7f6052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:20.836876 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e554d78-7e98-419a-bf41-74b4bc7f6052 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:20.845536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1285/2540] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:20.852057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb2eaf-830a-4fb1-8541-46ad44391e94 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:20.869429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb2eaf-830a-4fb1-8541-46ad44391e94 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:20.876329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a6aee-6c47-4122-b104-5673d04d1b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:20.877977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a6aee-6c47-4122-b104-5673d04d1b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:20.878272 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a6aee-6c47-4122-b104-5673d04d1b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:20.878590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a6aee-6c47-4122-b104-5673d04d1b1e admin admin] PATH_INFO: `/v3/projects/3ff390033801478cb103528955387ced/users/452a6d0dac1446d495978928ebe5dfc7/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:20.902854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a6aee-6c47-4122-b104-5673d04d1b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff390033801478cb103528955387ced, user_id=452a6d0dac1446d495978928ebe5dfc7, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:20.903610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a6aee-6c47-4122-b104-5673d04d1b1e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:20.986533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1256/2541] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:20 2022] PUT /identity/v3/projects/3ff390033801478cb103528955387ced/users/452a6d0dac1446d495978928ebe5dfc7/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:20.991073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e554d78-7e98-419a-bf41-74b4bc7f6052 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:21.004053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e554d78-7e98-419a-bf41-74b4bc7f6052 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:21.022417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:21.077542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb44d5f-cb0b-4581-81e2-0f87b5d62c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:21.078310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44d5f-cb0b-4581-81e2-0f87b5d62c9c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:21.078593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44d5f-cb0b-4581-81e2-0f87b5d62c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:21.078867 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb44d5f-cb0b-4581-81e2-0f87b5d62c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:21.079615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44d5f-cb0b-4581-81e2-0f87b5d62c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:21.081204 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb44d5f-cb0b-4581-81e2-0f87b5d62c9c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:21.091447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1286/2542] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:21.095509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a6aee-6c47-4122-b104-5673d04d1b1e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:21.107262 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a6aee-6c47-4122-b104-5673d04d1b1e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:21.112215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09e708-2504-4baa-8369-b199717353c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:21.113258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09e708-2504-4baa-8369-b199717353c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:21.113730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09e708-2504-4baa-8369-b199717353c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:21.114106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09e708-2504-4baa-8369-b199717353c7 admin admin] PATH_INFO: `/v3/projects/3ff390033801478cb103528955387ced/users/452a6d0dac1446d495978928ebe5dfc7/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:21.116232 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09e708-2504-4baa-8369-b199717353c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff390033801478cb103528955387ced, user_id=452a6d0dac1446d495978928ebe5dfc7, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:21.119067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09e708-2504-4baa-8369-b199717353c7 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:21.129554 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1257/2543] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:21 2022] PUT /identity/v3/projects/3ff390033801478cb103528955387ced/users/452a6d0dac1446d495978928ebe5dfc7/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:21.141090 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499e51c-d5ff-419e-8bb6-4af69afebe81 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:21.141379 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499e51c-d5ff-419e-8bb6-4af69afebe81 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:21.141660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499e51c-d5ff-419e-8bb6-4af69afebe81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:21.169493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9499e51c-d5ff-419e-8bb6-4af69afebe81 None None] MFA Rules not processed for user `452a6d0dac1446d495978928ebe5dfc7`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:21.170798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9499e51c-d5ff-419e-8bb6-4af69afebe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:21.241470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1287/2544] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:21 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:21.252693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09e708-2504-4baa-8369-b199717353c7 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:21.254935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9f09e708-2504-4baa-8369-b199717353c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:21.319631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09e708-2504-4baa-8369-b199717353c7 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:21.323469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99efe7a-90dc-4b6b-b97d-5e19f0f59f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:21.324120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99efe7a-90dc-4b6b-b97d-5e19f0f59f09 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:21.324404 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99efe7a-90dc-4b6b-b97d-5e19f0f59f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:21.324681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99efe7a-90dc-4b6b-b97d-5e19f0f59f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:21.335378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99efe7a-90dc-4b6b-b97d-5e19f0f59f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:21.336861 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99efe7a-90dc-4b6b-b97d-5e19f0f59f09 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:21.354037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1258/2545] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:21 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:21.549211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499e51c-d5ff-419e-8bb6-4af69afebe81 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:21.559855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499e51c-d5ff-419e-8bb6-4af69afebe81 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:21.563821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad898d-1638-426e-8c8b-23839b387832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:21.564611 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad898d-1638-426e-8c8b-23839b387832 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:21.564967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad898d-1638-426e-8c8b-23839b387832 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:21.565314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad898d-1638-426e-8c8b-23839b387832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:21.627433 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad898d-1638-426e-8c8b-23839b387832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:21.628856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad898d-1638-426e-8c8b-23839b387832 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:21.640673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1288/2546] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:25.097057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99efe7a-90dc-4b6b-b97d-5e19f0f59f09 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:25.157866 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99efe7a-90dc-4b6b-b97d-5e19f0f59f09 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:25.162986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:25.163848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:25.164130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:25.164406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] PATH_INFO: `/v3/users/f6c65710fce44d12a8d559431ac866a8` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:25.165570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c65710fce44d12a8d559431ac866a8)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:25.167578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:25.317862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c65710fce44d12a8d559431ac866a8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:25.318606 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c65710fce44d12a8d559431ac866a8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:25.323004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:25.331406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c65710fce44d12a8d559431ac866a8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:25.338126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c65710fce44d12a8d559431ac866a8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:25.408058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c65710fce44d12a8d559431ac866a8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:25.415683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1259/2547] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:25 2022] DELETE /identity/v3/users/f6c65710fce44d12a8d559431ac866a8 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:05:25.420263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308474eb-d064-4e07-b5a2-dc77f2455885 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:25.420732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308474eb-d064-4e07-b5a2-dc77f2455885 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:25.420832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308474eb-d064-4e07-b5a2-dc77f2455885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:25.437743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-308474eb-d064-4e07-b5a2-dc77f2455885 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:25.438980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-308474eb-d064-4e07-b5a2-dc77f2455885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:25.471248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:25.498586 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1289/2548] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:25.508137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:25.509683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:25.565806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b68fd4-429c-45d1-8c41-62d27922c02f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:25.569500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42996887-fcd3-4e9c-a7fc-3a7bd5e3924f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:25.570176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42996887-fcd3-4e9c-a7fc-3a7bd5e3924f service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:25.570462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42996887-fcd3-4e9c-a7fc-3a7bd5e3924f service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:25.570741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42996887-fcd3-4e9c-a7fc-3a7bd5e3924f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:25.579942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42996887-fcd3-4e9c-a7fc-3a7bd5e3924f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:25.581392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42996887-fcd3-4e9c-a7fc-3a7bd5e3924f service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:25.591994 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1260/2549] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:25.681933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308474eb-d064-4e07-b5a2-dc77f2455885 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:25.691514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308474eb-d064-4e07-b5a2-dc77f2455885 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:25.695088 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d5761-0992-4fa0-a0c4-58659c47f187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:25.695727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d5761-0992-4fa0-a0c4-58659c47f187 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:25.696010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d5761-0992-4fa0-a0c4-58659c47f187 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:25.696288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d5761-0992-4fa0-a0c4-58659c47f187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:25.698087 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b90d5761-0992-4fa0-a0c4-58659c47f187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:25.710436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d5761-0992-4fa0-a0c4-58659c47f187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:25.711856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d5761-0992-4fa0-a0c4-58659c47f187 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:25.722045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1290/2550] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:26.049661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42996887-fcd3-4e9c-a7fc-3a7bd5e3924f service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:26.052039 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-42996887-fcd3-4e9c-a7fc-3a7bd5e3924f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:26.071326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42996887-fcd3-4e9c-a7fc-3a7bd5e3924f service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:26.076351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:26.077222 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:26.077506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:26.077792 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] PATH_INFO: `/v3/projects/6df38c9fa8264a229fc71c49bd9143ab` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:26.079004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df38c9fa8264a229fc71c49bd9143ab)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:26.080992 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:26.187998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df38c9fa8264a229fc71c49bd9143ab', 'request_id': 'req-a4ed0112-5253-46ae-9078-fefe9e34e27e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.194156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df38c9fa8264a229fc71c49bd9143ab', 'request_id': 'req-a4ed0112-5253-46ae-9078-fefe9e34e27e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.288076 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df38c9fa8264a229fc71c49bd9143ab', 'request_id': 'req-a4ed0112-5253-46ae-9078-fefe9e34e27e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.299895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] The token cache is being invalidate because project 6df38c9fa8264a229fc71c49bd9143ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:26.300175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.301329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.302986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1261/2551] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:26 2022] DELETE /identity/v3/projects/6df38c9fa8264a229fc71c49bd9143ab => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:26.306936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d5761-0992-4fa0-a0c4-58659c47f187 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:26.317341 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d5761-0992-4fa0-a0c4-58659c47f187 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:26.373917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:26.374853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:26.375138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:26.375419 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] PATH_INFO: `/v3/users/452a6d0dac1446d495978928ebe5dfc7` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:26.376682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=452a6d0dac1446d495978928ebe5dfc7)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:26.378710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:26.524919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452a6d0dac1446d495978928ebe5dfc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.525727 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452a6d0dac1446d495978928ebe5dfc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.533025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452a6d0dac1446d495978928ebe5dfc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.545765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452a6d0dac1446d495978928ebe5dfc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.648377 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-8d654a9c-381d-4476-94f6-1e0693fb7c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452a6d0dac1446d495978928ebe5dfc7'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:26.655147 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1291/2552] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:26 2022] DELETE /identity/v3/users/452a6d0dac1446d495978928ebe5dfc7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:27.056654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:27.058469 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:27.078709 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed0112-5253-46ae-9078-fefe9e34e27e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:27.139177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:27.146253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:27.147898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.148283 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.148618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] PATH_INFO: `/v3/projects/3ff390033801478cb103528955387ced` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.150125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff390033801478cb103528955387ced)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:27.152927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:27.243363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff390033801478cb103528955387ced', 'request_id': 'req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:27.248790 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff390033801478cb103528955387ced', 'request_id': 'req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:27.288127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff390033801478cb103528955387ced', 'request_id': 'req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:27.302025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] The token cache is being invalidate because project 3ff390033801478cb103528955387ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:27.302360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:27.303020 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-378c6e2b-06e1-4c70-97fb-8cbb386d9d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:27.304663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1262/2553] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:27 2022] DELETE /identity/v3/projects/3ff390033801478cb103528955387ced => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:27.309638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865ec3-0b0e-4ab4-b629-1fd4dd745b45 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.310031 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865ec3-0b0e-4ab4-b629-1fd4dd745b45 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.310382 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865ec3-0b0e-4ab4-b629-1fd4dd745b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.328764 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e7865ec3-0b0e-4ab4-b629-1fd4dd745b45 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:27.330003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e7865ec3-0b0e-4ab4-b629-1fd4dd745b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:27.372564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:27.408041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1292/2554] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:27.410007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591494d2-5bbe-4c40-835c-65f7cecc2a42 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.410834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591494d2-5bbe-4c40-835c-65f7cecc2a42 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.411140 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591494d2-5bbe-4c40-835c-65f7cecc2a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.430370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-591494d2-5bbe-4c40-835c-65f7cecc2a42 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:27.431643 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-591494d2-5bbe-4c40-835c-65f7cecc2a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:27.440111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1263/2555] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:27.444149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7865ec3-0b0e-4ab4-b629-1fd4dd745b45 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:27.453735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7865ec3-0b0e-4ab4-b629-1fd4dd745b45 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:27.458473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94f639-ffa4-4076-895f-76f71fbd714d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:27.459123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94f639-ffa4-4076-895f-76f71fbd714d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.459404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94f639-ffa4-4076-895f-76f71fbd714d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.459680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94f639-ffa4-4076-895f-76f71fbd714d admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.460396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94f639-ffa4-4076-895f-76f71fbd714d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:05:27.460699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94f639-ffa4-4076-895f-76f71fbd714d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:27.462316 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94f639-ffa4-4076-895f-76f71fbd714d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:27.477326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1293/2556] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:05:27.479631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591494d2-5bbe-4c40-835c-65f7cecc2a42 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:27.489383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591494d2-5bbe-4c40-835c-65f7cecc2a42 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:27.494148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac952dc-c3e4-4130-9c95-01b6650810fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:27.494800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac952dc-c3e4-4130-9c95-01b6650810fb admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.495081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac952dc-c3e4-4130-9c95-01b6650810fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.495355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac952dc-c3e4-4130-9c95-01b6650810fb admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.496044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac952dc-c3e4-4130-9c95-01b6650810fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:27.498002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac952dc-c3e4-4130-9c95-01b6650810fb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:27.615585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1264/2557] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:27 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:27.616787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94f639-ffa4-4076-895f-76f71fbd714d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:27.627631 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94f639-ffa4-4076-895f-76f71fbd714d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:27.688413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79767c-8c87-40be-ab70-f003683471ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:27.689459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79767c-8c87-40be-ab70-f003683471ac admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.689815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79767c-8c87-40be-ab70-f003683471ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.690594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79767c-8c87-40be-ab70-f003683471ac admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.690950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79767c-8c87-40be-ab70-f003683471ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:27.693117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79767c-8c87-40be-ab70-f003683471ac admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:27.760470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1294/2558] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:27 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:27.762089 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac952dc-c3e4-4130-9c95-01b6650810fb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:27.773125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac952dc-c3e4-4130-9c95-01b6650810fb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:27.777935 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fef68-a3e3-4a5d-a93f-5195c7af026e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:27.778631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fef68-a3e3-4a5d-a93f-5195c7af026e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.778983 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fef68-a3e3-4a5d-a93f-5195c7af026e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.779346 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fef68-a3e3-4a5d-a93f-5195c7af026e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.780164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fef68-a3e3-4a5d-a93f-5195c7af026e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:27.781980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fef68-a3e3-4a5d-a93f-5195c7af026e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:27.793696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1265/2559] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:27.797782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79767c-8c87-40be-ab70-f003683471ac admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:27.808556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79767c-8c87-40be-ab70-f003683471ac admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:27.813830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc367c-69d6-498e-8b90-84ec3ceb3ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:27.814732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc367c-69d6-498e-8b90-84ec3ceb3ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.815026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc367c-69d6-498e-8b90-84ec3ceb3ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.815308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc367c-69d6-498e-8b90-84ec3ceb3ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.816046 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc367c-69d6-498e-8b90-84ec3ceb3ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:27.817687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc367c-69d6-498e-8b90-84ec3ceb3ff1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:27.829550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1295/2560] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:27.834429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fef68-a3e3-4a5d-a93f-5195c7af026e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:27.846267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fef68-a3e3-4a5d-a93f-5195c7af026e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:27.854102 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef84b7-c0fa-475b-8e78-1c7292bfde6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:27.855119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef84b7-c0fa-475b-8e78-1c7292bfde6b admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.855652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef84b7-c0fa-475b-8e78-1c7292bfde6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.856028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef84b7-c0fa-475b-8e78-1c7292bfde6b admin admin] PATH_INFO: `/v3/projects/698b89f5dc604e568b1c133eda7b2804/users/39aa02a6acfb4ade9c6093908ec98084/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.873706 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef84b7-c0fa-475b-8e78-1c7292bfde6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=698b89f5dc604e568b1c133eda7b2804, user_id=39aa02a6acfb4ade9c6093908ec98084, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:27.876795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef84b7-c0fa-475b-8e78-1c7292bfde6b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:27.945243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1266/2561] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:27 2022] PUT /identity/v3/projects/698b89f5dc604e568b1c133eda7b2804/users/39aa02a6acfb4ade9c6093908ec98084/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:27.955643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a981928-216f-43e5-a211-8a1da1351b2e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:27.955932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a981928-216f-43e5-a211-8a1da1351b2e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:27.956211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a981928-216f-43e5-a211-8a1da1351b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:27.986028 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0a981928-216f-43e5-a211-8a1da1351b2e None None] MFA Rules not processed for user `39aa02a6acfb4ade9c6093908ec98084`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:27.987241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0a981928-216f-43e5-a211-8a1da1351b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:28.074813 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1296/2562] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:28.079407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef84b7-c0fa-475b-8e78-1c7292bfde6b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:28.141986 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef84b7-c0fa-475b-8e78-1c7292bfde6b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:28.147375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa382d6-3b6f-498f-a2f4-9999132564f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:28.148114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa382d6-3b6f-498f-a2f4-9999132564f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:28.148356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa382d6-3b6f-498f-a2f4-9999132564f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:28.148634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa382d6-3b6f-498f-a2f4-9999132564f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:28.149314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa382d6-3b6f-498f-a2f4-9999132564f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:28.151251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa382d6-3b6f-498f-a2f4-9999132564f0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:28.227364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1267/2563] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:28 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:28.231126 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a981928-216f-43e5-a211-8a1da1351b2e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:28.242282 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a981928-216f-43e5-a211-8a1da1351b2e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:28.283473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:28.301301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e58aed-6cfe-485b-b068-abdd6ae301f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:28.302015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e58aed-6cfe-485b-b068-abdd6ae301f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:28.302307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e58aed-6cfe-485b-b068-abdd6ae301f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:28.302585 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e58aed-6cfe-485b-b068-abdd6ae301f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:28.303328 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e58aed-6cfe-485b-b068-abdd6ae301f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:28.305713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e58aed-6cfe-485b-b068-abdd6ae301f4 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:28.413396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1297/2564] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:28 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:28.413526 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa382d6-3b6f-498f-a2f4-9999132564f0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:28.423533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa382d6-3b6f-498f-a2f4-9999132564f0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:28.428742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790839b-1005-4a9e-8f05-b666e3640609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:28.429435 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790839b-1005-4a9e-8f05-b666e3640609 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:28.429719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790839b-1005-4a9e-8f05-b666e3640609 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:28.430033 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790839b-1005-4a9e-8f05-b666e3640609 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:28.430753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790839b-1005-4a9e-8f05-b666e3640609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:28.432353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790839b-1005-4a9e-8f05-b666e3640609 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:28.433820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:28.445332 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1268/2565] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:28.449411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e58aed-6cfe-485b-b068-abdd6ae301f4 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:28.459722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e58aed-6cfe-485b-b068-abdd6ae301f4 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:28.464245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1ed14-f2fe-437f-ae27-b24a6f0f7dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:28.465113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1ed14-f2fe-437f-ae27-b24a6f0f7dca admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:28.465395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1ed14-f2fe-437f-ae27-b24a6f0f7dca admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:28.465698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1ed14-f2fe-437f-ae27-b24a6f0f7dca admin admin] PATH_INFO: `/v3/projects/d1f4e80590b94eeda8f096e635d845c3/users/12a69fc482a34a5fa7c34f9b646437a5/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:28.481081 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1ed14-f2fe-437f-ae27-b24a6f0f7dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f4e80590b94eeda8f096e635d845c3, user_id=12a69fc482a34a5fa7c34f9b646437a5, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:28.483750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1ed14-f2fe-437f-ae27-b24a6f0f7dca admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:28.585704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1298/2566] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:28 2022] PUT /identity/v3/projects/d1f4e80590b94eeda8f096e635d845c3/users/12a69fc482a34a5fa7c34f9b646437a5/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:28.589908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790839b-1005-4a9e-8f05-b666e3640609 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:28.600278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790839b-1005-4a9e-8f05-b666e3640609 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:28.657822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5f6f3-b6e1-48b1-8253-48e23177d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:28.658542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5f6f3-b6e1-48b1-8253-48e23177d5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:28.658860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5f6f3-b6e1-48b1-8253-48e23177d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:28.659136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5f6f3-b6e1-48b1-8253-48e23177d5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:28.659888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5f6f3-b6e1-48b1-8253-48e23177d5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:28.661514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5f6f3-b6e1-48b1-8253-48e23177d5d8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:28.671468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1269/2567] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:28.676210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1ed14-f2fe-437f-ae27-b24a6f0f7dca admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:28.686766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1ed14-f2fe-437f-ae27-b24a6f0f7dca admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:28.692038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0f95c-89d3-4ce3-b8b1-34013eebaab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:28.693115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0f95c-89d3-4ce3-b8b1-34013eebaab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:28.693376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0f95c-89d3-4ce3-b8b1-34013eebaab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:28.693651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0f95c-89d3-4ce3-b8b1-34013eebaab0 admin admin] PATH_INFO: `/v3/projects/d1f4e80590b94eeda8f096e635d845c3/users/12a69fc482a34a5fa7c34f9b646437a5/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:28.695658 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0f95c-89d3-4ce3-b8b1-34013eebaab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f4e80590b94eeda8f096e635d845c3, user_id=12a69fc482a34a5fa7c34f9b646437a5, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:28.698358 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0f95c-89d3-4ce3-b8b1-34013eebaab0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:28.709600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1299/2568] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:28 2022] PUT /identity/v3/projects/d1f4e80590b94eeda8f096e635d845c3/users/12a69fc482a34a5fa7c34f9b646437a5/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:28.720752 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84d0c4-85e4-4426-b4c9-8bb15f7dac48 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:28.721045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84d0c4-85e4-4426-b4c9-8bb15f7dac48 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:28.721326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84d0c4-85e4-4426-b4c9-8bb15f7dac48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:28.750871 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2f84d0c4-85e4-4426-b4c9-8bb15f7dac48 None None] MFA Rules not processed for user `12a69fc482a34a5fa7c34f9b646437a5`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:28.754365 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2f84d0c4-85e4-4426-b4c9-8bb15f7dac48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:28.823688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1270/2569] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:28.834667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0f95c-89d3-4ce3-b8b1-34013eebaab0 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:28.836436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b5c0f95c-89d3-4ce3-b8b1-34013eebaab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:28.912534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0f95c-89d3-4ce3-b8b1-34013eebaab0 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:28.916461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f5e54-596c-4a5e-827f-6064682ddce9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:28.919842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f5e54-596c-4a5e-827f-6064682ddce9 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:28.919842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f5e54-596c-4a5e-827f-6064682ddce9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:28.919842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f5e54-596c-4a5e-827f-6064682ddce9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:28.992092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f5e54-596c-4a5e-827f-6064682ddce9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:28.993574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f5e54-596c-4a5e-827f-6064682ddce9 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:29.005013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1300/2570] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:29.672681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84d0c4-85e4-4426-b4c9-8bb15f7dac48 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:29.674495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2f84d0c4-85e4-4426-b4c9-8bb15f7dac48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:29.740260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84d0c4-85e4-4426-b4c9-8bb15f7dac48 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:29.744693 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:29.745556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:29.745838 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:29.746139 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] PATH_INFO: `/v3/users/5a304d58eff84a87953dc7af6a0afe87` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:29.747305 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a304d58eff84a87953dc7af6a0afe87)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:29.749205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:29.773455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:29.998758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a304d58eff84a87953dc7af6a0afe87'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:29.999326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a304d58eff84a87953dc7af6a0afe87'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:30.011427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a304d58eff84a87953dc7af6a0afe87'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:30.018310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a304d58eff84a87953dc7af6a0afe87'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:30.093537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0b113a2f-4060-40a9-ba3b-762b637dc189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a304d58eff84a87953dc7af6a0afe87'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:30.100510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1271/2571] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:29 2022] DELETE /identity/v3/users/5a304d58eff84a87953dc7af6a0afe87 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:30.318273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f5e54-596c-4a5e-827f-6064682ddce9 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:30.330090 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f5e54-596c-4a5e-827f-6064682ddce9 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:30.375417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:30.388713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:30.389627 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:30.389941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:30.390221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] PATH_INFO: `/v3/projects/57756244f164417dad785bb5927d05b6` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:30.391422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57756244f164417dad785bb5927d05b6)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:30.393343 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:30.498529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57756244f164417dad785bb5927d05b6', 'request_id': 'req-47eea6f5-1283-40aa-9f2a-9b3973d41414'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:30.503962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57756244f164417dad785bb5927d05b6', 'request_id': 'req-47eea6f5-1283-40aa-9f2a-9b3973d41414'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:30.569766 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57756244f164417dad785bb5927d05b6', 'request_id': 'req-47eea6f5-1283-40aa-9f2a-9b3973d41414'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:30.580987 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] The token cache is being invalidate because project 57756244f164417dad785bb5927d05b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:30.581273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:30.581907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-47eea6f5-1283-40aa-9f2a-9b3973d41414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:30.584072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1301/2572] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:30 2022] DELETE /identity/v3/projects/57756244f164417dad785bb5927d05b6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:30.588960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6aa74c-9711-4cc2-805d-bbac370ca833 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:30.589260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6aa74c-9711-4cc2-805d-bbac370ca833 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:30.589545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6aa74c-9711-4cc2-805d-bbac370ca833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:30.607195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bd6aa74c-9711-4cc2-805d-bbac370ca833 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:30.608394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bd6aa74c-9711-4cc2-805d-bbac370ca833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:30.683593 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1272/2573] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:30.691834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03719-3899-4ce3-b888-c38b416ec8ed None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:30.709557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03719-3899-4ce3-b888-c38b416ec8ed None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:30.709557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f03719-3899-4ce3-b888-c38b416ec8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:30.713183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-80f03719-3899-4ce3-b888-c38b416ec8ed None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:30.714418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-80f03719-3899-4ce3-b888-c38b416ec8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:30.722640 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1302/2574] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:30.727238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6aa74c-9711-4cc2-805d-bbac370ca833 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:30.737529 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6aa74c-9711-4cc2-805d-bbac370ca833 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:30.742767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112a15-8ed5-4094-9d1f-37c11cb07779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:30.743471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112a15-8ed5-4094-9d1f-37c11cb07779 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:30.743740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112a15-8ed5-4094-9d1f-37c11cb07779 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:30.744040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112a15-8ed5-4094-9d1f-37c11cb07779 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:30.744767 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c112a15-8ed5-4094-9d1f-37c11cb07779 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:05:30.745076 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c112a15-8ed5-4094-9d1f-37c11cb07779 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:30.746720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c112a15-8ed5-4094-9d1f-37c11cb07779 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:30.759836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1273/2575] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:05:30.764168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f03719-3899-4ce3-b888-c38b416ec8ed None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:30.773352 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f03719-3899-4ce3-b888-c38b416ec8ed None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:30.778034 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64db53-ea3e-4ae5-9dcb-1ca6181ef654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:30.778690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64db53-ea3e-4ae5-9dcb-1ca6181ef654 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:30.778957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64db53-ea3e-4ae5-9dcb-1ca6181ef654 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:30.779233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64db53-ea3e-4ae5-9dcb-1ca6181ef654 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:30.779923 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64db53-ea3e-4ae5-9dcb-1ca6181ef654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:30.781834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64db53-ea3e-4ae5-9dcb-1ca6181ef654 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:30.868164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1303/2576] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:30 2022] POST /identity/v3/projects => generated 357 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:30.872210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112a15-8ed5-4094-9d1f-37c11cb07779 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:30.883329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112a15-8ed5-4094-9d1f-37c11cb07779 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:30.943154 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a073775-3b4b-4b5f-816a-0249580cf902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:30.945809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a073775-3b4b-4b5f-816a-0249580cf902 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:30.945809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a073775-3b4b-4b5f-816a-0249580cf902 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:30.945809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a073775-3b4b-4b5f-816a-0249580cf902 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:30.945809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a073775-3b4b-4b5f-816a-0249580cf902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:30.947059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a073775-3b4b-4b5f-816a-0249580cf902 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.054694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1274/2577] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:30 2022] POST /identity/v3/users => generated 279 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:31.058694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64db53-ea3e-4ae5-9dcb-1ca6181ef654 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:31.069185 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64db53-ea3e-4ae5-9dcb-1ca6181ef654 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:31.073865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ffab9-fa7c-46d8-8398-c86388b66f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:31.074544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ffab9-fa7c-46d8-8398-c86388b66f36 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.074828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ffab9-fa7c-46d8-8398-c86388b66f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.075108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ffab9-fa7c-46d8-8398-c86388b66f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.075831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ffab9-fa7c-46d8-8398-c86388b66f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:31.077414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ffab9-fa7c-46d8-8398-c86388b66f36 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.088490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1304/2578] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:31.092384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a073775-3b4b-4b5f-816a-0249580cf902 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:31.102949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a073775-3b4b-4b5f-816a-0249580cf902 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:31.107796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fbba9-295e-4b05-8a91-8468ca489f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:31.108489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fbba9-295e-4b05-8a91-8468ca489f82 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.108776 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fbba9-295e-4b05-8a91-8468ca489f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.109058 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fbba9-295e-4b05-8a91-8468ca489f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.109824 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fbba9-295e-4b05-8a91-8468ca489f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:31.111506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fbba9-295e-4b05-8a91-8468ca489f82 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.122527 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1275/2579] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:31.126557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ffab9-fa7c-46d8-8398-c86388b66f36 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:31.136726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ffab9-fa7c-46d8-8398-c86388b66f36 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:31.141374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60490186-94e4-4b5f-85e9-1ea282c695a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:31.142285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60490186-94e4-4b5f-85e9-1ea282c695a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.142580 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60490186-94e4-4b5f-85e9-1ea282c695a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.142861 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60490186-94e4-4b5f-85e9-1ea282c695a6 admin admin] PATH_INFO: `/v3/projects/f033898c65bb4803a28414dd36fb55b6/users/ed16f49c1255493d8468f66f1d86adc0/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.157669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60490186-94e4-4b5f-85e9-1ea282c695a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f033898c65bb4803a28414dd36fb55b6, user_id=ed16f49c1255493d8468f66f1d86adc0, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:31.160353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60490186-94e4-4b5f-85e9-1ea282c695a6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.253214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1305/2580] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:31 2022] PUT /identity/v3/projects/f033898c65bb4803a28414dd36fb55b6/users/ed16f49c1255493d8468f66f1d86adc0/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:31.263691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5a7e2-35ff-4fa5-bb3d-b60dce226b8a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.263980 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5a7e2-35ff-4fa5-bb3d-b60dce226b8a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.264260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5a7e2-35ff-4fa5-bb3d-b60dce226b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.270037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:31.297572 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-48a5a7e2-35ff-4fa5-bb3d-b60dce226b8a None None] MFA Rules not processed for user `ed16f49c1255493d8468f66f1d86adc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:31.298860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-48a5a7e2-35ff-4fa5-bb3d-b60dce226b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:31.365335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1276/2581] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:31 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:31.369797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60490186-94e4-4b5f-85e9-1ea282c695a6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:31.433374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60490186-94e4-4b5f-85e9-1ea282c695a6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:31.438063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c24333-2955-4f6e-b9e0-74e81a5e00c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:31.438716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c24333-2955-4f6e-b9e0-74e81a5e00c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.439000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c24333-2955-4f6e-b9e0-74e81a5e00c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.439277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c24333-2955-4f6e-b9e0-74e81a5e00c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.439961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c24333-2955-4f6e-b9e0-74e81a5e00c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:31.441848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c24333-2955-4f6e-b9e0-74e81a5e00c2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.533613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:31.541339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1306/2582] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:31 2022] POST /identity/v3/projects => generated 355 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:31.545337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5a7e2-35ff-4fa5-bb3d-b60dce226b8a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:31.555884 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5a7e2-35ff-4fa5-bb3d-b60dce226b8a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:31.611611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d779e-a7ce-450c-9c04-1a95ec3ee992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:31.612410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d779e-a7ce-450c-9c04-1a95ec3ee992 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.612690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d779e-a7ce-450c-9c04-1a95ec3ee992 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.612990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d779e-a7ce-450c-9c04-1a95ec3ee992 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.613673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d779e-a7ce-450c-9c04-1a95ec3ee992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:31.615711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d779e-a7ce-450c-9c04-1a95ec3ee992 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.687722 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1277/2583] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:31 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:31.691698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c24333-2955-4f6e-b9e0-74e81a5e00c2 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:31.702317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c24333-2955-4f6e-b9e0-74e81a5e00c2 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:31.706892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e8fff-ec32-4202-a442-2d46f9974556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:31.707544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e8fff-ec32-4202-a442-2d46f9974556 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.708420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e8fff-ec32-4202-a442-2d46f9974556 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.708700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e8fff-ec32-4202-a442-2d46f9974556 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.709419 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e8fff-ec32-4202-a442-2d46f9974556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:31.711065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e8fff-ec32-4202-a442-2d46f9974556 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.724776 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1307/2584] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:31.728898 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d779e-a7ce-450c-9c04-1a95ec3ee992 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:31.739046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d779e-a7ce-450c-9c04-1a95ec3ee992 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:31.743669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dd1cf-9739-4597-854a-937955657ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:31.744542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dd1cf-9739-4597-854a-937955657ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.744829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dd1cf-9739-4597-854a-937955657ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.745108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dd1cf-9739-4597-854a-937955657ec5 admin admin] PATH_INFO: `/v3/projects/918b92eed82e4e98b20c4faf7d3b9eac/users/106b0ed792a64a42964a218644144559/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.760254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00dd1cf-9739-4597-854a-937955657ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918b92eed82e4e98b20c4faf7d3b9eac, user_id=106b0ed792a64a42964a218644144559, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:31.763022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00dd1cf-9739-4597-854a-937955657ec5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.847389 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1278/2585] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:31 2022] PUT /identity/v3/projects/918b92eed82e4e98b20c4faf7d3b9eac/users/106b0ed792a64a42964a218644144559/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:31.851347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e8fff-ec32-4202-a442-2d46f9974556 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:31.861226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e8fff-ec32-4202-a442-2d46f9974556 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:31.916120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c007cd-48b5-4d96-9fe8-55829442562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:31.916711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c007cd-48b5-4d96-9fe8-55829442562a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.917001 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c007cd-48b5-4d96-9fe8-55829442562a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.917282 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c007cd-48b5-4d96-9fe8-55829442562a admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.918042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c007cd-48b5-4d96-9fe8-55829442562a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:31.919625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c007cd-48b5-4d96-9fe8-55829442562a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.929013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1308/2586] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:31.933207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dd1cf-9739-4597-854a-937955657ec5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:31.943932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dd1cf-9739-4597-854a-937955657ec5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:31.948537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d22a6-1c5b-4e8a-bb45-7f1fc18e74e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:31.949409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d22a6-1c5b-4e8a-bb45-7f1fc18e74e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.949696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d22a6-1c5b-4e8a-bb45-7f1fc18e74e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.950019 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d22a6-1c5b-4e8a-bb45-7f1fc18e74e3 admin admin] PATH_INFO: `/v3/projects/918b92eed82e4e98b20c4faf7d3b9eac/users/106b0ed792a64a42964a218644144559/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:31.952007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d22a6-1c5b-4e8a-bb45-7f1fc18e74e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=918b92eed82e4e98b20c4faf7d3b9eac, user_id=106b0ed792a64a42964a218644144559, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:31.954785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d22a6-1c5b-4e8a-bb45-7f1fc18e74e3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:31.965468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1279/2587] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:31 2022] PUT /identity/v3/projects/918b92eed82e4e98b20c4faf7d3b9eac/users/106b0ed792a64a42964a218644144559/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:31.977152 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d994d-f574-42d3-b91f-0f1fa43ef0c6 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:31.977552 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d994d-f574-42d3-b91f-0f1fa43ef0c6 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:31.977819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d994d-f574-42d3-b91f-0f1fa43ef0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:32.004305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2b2d994d-f574-42d3-b91f-0f1fa43ef0c6 None None] MFA Rules not processed for user `106b0ed792a64a42964a218644144559`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:32.005508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2b2d994d-f574-42d3-b91f-0f1fa43ef0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:32.072601 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1309/2588] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:31 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:32.084741 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d22a6-1c5b-4e8a-bb45-7f1fc18e74e3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:32.086493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-752d22a6-1c5b-4e8a-bb45-7f1fc18e74e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:32.131699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:32.142390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d22a6-1c5b-4e8a-bb45-7f1fc18e74e3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:32.146731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25ffc4-a22e-4ab3-b5ef-4790bf933d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:32.147206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25ffc4-a22e-4ab3-b5ef-4790bf933d69 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:32.147492 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25ffc4-a22e-4ab3-b5ef-4790bf933d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:32.147774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25ffc4-a22e-4ab3-b5ef-4790bf933d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:32.156637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25ffc4-a22e-4ab3-b5ef-4790bf933d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:32.158092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25ffc4-a22e-4ab3-b5ef-4790bf933d69 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:32.168291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1280/2589] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:32 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:32.392135 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d994d-f574-42d3-b91f-0f1fa43ef0c6 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:32.402758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d994d-f574-42d3-b91f-0f1fa43ef0c6 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:32.406424 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4abc6-082d-4a3d-872f-3648d0c3a3e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:32.407061 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4abc6-082d-4a3d-872f-3648d0c3a3e2 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:32.407348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4abc6-082d-4a3d-872f-3648d0c3a3e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:32.407628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4abc6-082d-4a3d-872f-3648d0c3a3e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:32.464994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4abc6-082d-4a3d-872f-3648d0c3a3e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:32.466458 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4abc6-082d-4a3d-872f-3648d0c3a3e2 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:32.475281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1310/2590] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:32 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:35.432239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25ffc4-a22e-4ab3-b5ef-4790bf933d69 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:35.433629 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ba25ffc4-a22e-4ab3-b5ef-4790bf933d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:35.444845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25ffc4-a22e-4ab3-b5ef-4790bf933d69 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:35.448559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d799421-1ec2-46ee-94dd-101a565b7c06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:35.449297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d799421-1ec2-46ee-94dd-101a565b7c06 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:35.449648 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d799421-1ec2-46ee-94dd-101a565b7c06 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:35.449960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d799421-1ec2-46ee-94dd-101a565b7c06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:35.451716 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3d799421-1ec2-46ee-94dd-101a565b7c06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:35.516809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d799421-1ec2-46ee-94dd-101a565b7c06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:35.518301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d799421-1ec2-46ee-94dd-101a565b7c06 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:35.527972 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1281/2591] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:42.471188 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:45.523582 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=36 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:54.172715 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4abc6-082d-4a3d-872f-3648d0c3a3e2 service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:54.235195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4abc6-082d-4a3d-872f-3648d0c3a3e2 service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:54.239803 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:54.240665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:54.240963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:54.241235 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] PATH_INFO: `/v3/users/ed16f49c1255493d8468f66f1d86adc0` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:54.242442 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed16f49c1255493d8468f66f1d86adc0)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:54.244338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:54.437582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed16f49c1255493d8468f66f1d86adc0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:54.438261 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed16f49c1255493d8468f66f1d86adc0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:54.445053 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed16f49c1255493d8468f66f1d86adc0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:54.456869 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed16f49c1255493d8468f66f1d86adc0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:54.532872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed16f49c1255493d8468f66f1d86adc0'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:54.539066 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1311/2592] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:54 2022] DELETE /identity/v3/users/ed16f49c1255493d8468f66f1d86adc0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:54.544295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f71c0-7950-4639-98a8-ca816af464a8 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:54.544756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f71c0-7950-4639-98a8-ca816af464a8 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:54.544886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f71c0-7950-4639-98a8-ca816af464a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:54.561552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-8e4f71c0-7950-4639-98a8-ca816af464a8 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:54.562768 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8e4f71c0-7950-4639-98a8-ca816af464a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:54.621978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1282/2593] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:54.631385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:54.633052 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:54.690014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d9013-dbd3-4604-ab43-f6830a0b357a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:54.693556 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fce01-1455-417f-85c1-14a54b654040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:54.694231 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fce01-1455-417f-85c1-14a54b654040 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:54.694517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fce01-1455-417f-85c1-14a54b654040 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:54.694797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fce01-1455-417f-85c1-14a54b654040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:54.704074 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fce01-1455-417f-85c1-14a54b654040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:54.705659 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fce01-1455-417f-85c1-14a54b654040 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:54.716208 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1312/2594] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:05:55.129378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f71c0-7950-4639-98a8-ca816af464a8 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:55.130991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8e4f71c0-7950-4639-98a8-ca816af464a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:55.145941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f71c0-7950-4639-98a8-ca816af464a8 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:55.152556 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:55.153522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:55.153764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:55.154110 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] PATH_INFO: `/v3/projects/f033898c65bb4803a28414dd36fb55b6` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:55.156196 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f033898c65bb4803a28414dd36fb55b6)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:55.161840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:55.288937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f033898c65bb4803a28414dd36fb55b6', 'request_id': 'req-27e3116d-8226-491c-862f-d89296217c50'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.295063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f033898c65bb4803a28414dd36fb55b6', 'request_id': 'req-27e3116d-8226-491c-862f-d89296217c50'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.350627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f033898c65bb4803a28414dd36fb55b6', 'request_id': 'req-27e3116d-8226-491c-862f-d89296217c50'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.363107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] The token cache is being invalidate because project f033898c65bb4803a28414dd36fb55b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:55.363398 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.364066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.365647 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1283/2595] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:55 2022] DELETE /identity/v3/projects/f033898c65bb4803a28414dd36fb55b6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:55.369654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fce01-1455-417f-85c1-14a54b654040 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:55.379908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fce01-1455-417f-85c1-14a54b654040 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:55.398872 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:55.437920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:55.438848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:55.439128 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:55.439405 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] PATH_INFO: `/v3/users/106b0ed792a64a42964a218644144559` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:55.440623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=106b0ed792a64a42964a218644144559)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:55.442626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:55.639245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106b0ed792a64a42964a218644144559'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.639921 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106b0ed792a64a42964a218644144559'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.647434 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106b0ed792a64a42964a218644144559'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.659698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106b0ed792a64a42964a218644144559'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.727588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-e3a269c3-434b-4f73-8af1-81820c9f25d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106b0ed792a64a42964a218644144559'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:55.885972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1313/2596] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:05:55 2022] DELETE /identity/v3/users/106b0ed792a64a42964a218644144559 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:56.209384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:56.210965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:56.224013 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3116d-8226-491c-862f-d89296217c50 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:56.280666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:56.281521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:56.281808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:56.282118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] PATH_INFO: `/v3/projects/918b92eed82e4e98b20c4faf7d3b9eac` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:56.283280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=918b92eed82e4e98b20c4faf7d3b9eac)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:56.285190 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:56.374177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:56.390280 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918b92eed82e4e98b20c4faf7d3b9eac', 'request_id': 'req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:56.395950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918b92eed82e4e98b20c4faf7d3b9eac', 'request_id': 'req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:56.447080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918b92eed82e4e98b20c4faf7d3b9eac', 'request_id': 'req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:56.458137 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] The token cache is being invalidate because project 918b92eed82e4e98b20c4faf7d3b9eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:05:56.458533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:56.459246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-16b2bbdc-8c9d-45a1-8c32-08eeee110d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:05:56.461048 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1284/2597] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:05:56 2022] DELETE /identity/v3/projects/918b92eed82e4e98b20c4faf7d3b9eac => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:56.466139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78bf3e-d80c-4468-a33a-6c95681cfd8a None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:56.466630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78bf3e-d80c-4468-a33a-6c95681cfd8a None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:56.466749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78bf3e-d80c-4468-a33a-6c95681cfd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:56.485527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-bd78bf3e-d80c-4468-a33a-6c95681cfd8a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:56.486833 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bd78bf3e-d80c-4468-a33a-6c95681cfd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:56.545711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1314/2598] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:56.553520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e07ba-fd15-4a77-978d-4ae10b4f6530 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:56.553814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e07ba-fd15-4a77-978d-4ae10b4f6530 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:56.554135 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e07ba-fd15-4a77-978d-4ae10b4f6530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:56.572667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1d6e07ba-fd15-4a77-978d-4ae10b4f6530 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:56.573904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1d6e07ba-fd15-4a77-978d-4ae10b4f6530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:56.581861 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1285/2599] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:05:56.585981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78bf3e-d80c-4468-a33a-6c95681cfd8a None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:56.595670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78bf3e-d80c-4468-a33a-6c95681cfd8a None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:56.600300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc034b-358b-4fc0-b6c8-ad0ef4f0936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:56.600955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc034b-358b-4fc0-b6c8-ad0ef4f0936f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:56.601239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc034b-358b-4fc0-b6c8-ad0ef4f0936f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:56.601550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc034b-358b-4fc0-b6c8-ad0ef4f0936f admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:56.602320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc034b-358b-4fc0-b6c8-ad0ef4f0936f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:05:56.602632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc034b-358b-4fc0-b6c8-ad0ef4f0936f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:56.604222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc034b-358b-4fc0-b6c8-ad0ef4f0936f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:56.616294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1315/2600] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:05:56.621136 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e07ba-fd15-4a77-978d-4ae10b4f6530 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:56.630164 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e07ba-fd15-4a77-978d-4ae10b4f6530 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:56.635108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2ef5-2e6c-49f3-9ea6-ee98239413f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:56.635746 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2ef5-2e6c-49f3-9ea6-ee98239413f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:56.636078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2ef5-2e6c-49f3-9ea6-ee98239413f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:56.636360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2ef5-2e6c-49f3-9ea6-ee98239413f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:56.637036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d2ef5-2e6c-49f3-9ea6-ee98239413f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:56.638989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d2ef5-2e6c-49f3-9ea6-ee98239413f5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:56.740126 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1286/2601] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:05:56 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:56.744387 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc034b-358b-4fc0-b6c8-ad0ef4f0936f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:56.754410 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc034b-358b-4fc0-b6c8-ad0ef4f0936f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:56.781414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:05:56.813165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a676774-3643-4020-8f21-a0d9db2d0918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:56.813849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a676774-3643-4020-8f21-a0d9db2d0918 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:56.814371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a676774-3643-4020-8f21-a0d9db2d0918 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:56.815102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a676774-3643-4020-8f21-a0d9db2d0918 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:56.815909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a676774-3643-4020-8f21-a0d9db2d0918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:56.817958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a676774-3643-4020-8f21-a0d9db2d0918 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:56.913870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1316/2602] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:05:56 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:05:56.915134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2ef5-2e6c-49f3-9ea6-ee98239413f5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:56.926295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2ef5-2e6c-49f3-9ea6-ee98239413f5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:56.930944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a0eb5-304b-42c8-b402-b1e95abbd8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:56.931684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a0eb5-304b-42c8-b402-b1e95abbd8cd admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:56.932004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a0eb5-304b-42c8-b402-b1e95abbd8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:56.932289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a0eb5-304b-42c8-b402-b1e95abbd8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:56.933014 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a0eb5-304b-42c8-b402-b1e95abbd8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:56.934651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a0eb5-304b-42c8-b402-b1e95abbd8cd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:56.946279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1287/2603] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:05:56.950469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a676774-3643-4020-8f21-a0d9db2d0918 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:56.960612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a676774-3643-4020-8f21-a0d9db2d0918 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:56.965663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef716c-be9f-4b1d-bd70-835a0f862c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:56.966347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef716c-be9f-4b1d-bd70-835a0f862c82 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:56.966635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef716c-be9f-4b1d-bd70-835a0f862c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:56.966915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef716c-be9f-4b1d-bd70-835a0f862c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:56.967670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef716c-be9f-4b1d-bd70-835a0f862c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:56.969323 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef716c-be9f-4b1d-bd70-835a0f862c82 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:56.979009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1317/2604] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:05:56.983149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a0eb5-304b-42c8-b402-b1e95abbd8cd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:56.993043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a0eb5-304b-42c8-b402-b1e95abbd8cd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:56.998253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad668b-f817-49f3-bb07-e75c69dd2ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:56.999124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad668b-f817-49f3-bb07-e75c69dd2ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:56.999408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad668b-f817-49f3-bb07-e75c69dd2ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:56.999685 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad668b-f817-49f3-bb07-e75c69dd2ce1 admin admin] PATH_INFO: `/v3/projects/6a5a45705c3c4280bd89956c75e03ee1/users/759f7e79147b420b9a757d8819432027/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:57.014901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad668b-f817-49f3-bb07-e75c69dd2ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5a45705c3c4280bd89956c75e03ee1, user_id=759f7e79147b420b9a757d8819432027, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:57.017533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad668b-f817-49f3-bb07-e75c69dd2ce1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:57.087177 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1288/2605] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:05:56 2022] PUT /identity/v3/projects/6a5a45705c3c4280bd89956c75e03ee1/users/759f7e79147b420b9a757d8819432027/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:05:57.098485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f87a8-cfa6-4550-8a1f-534ccb654cbc None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:57.098779 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f87a8-cfa6-4550-8a1f-534ccb654cbc None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:57.099060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f87a8-cfa6-4550-8a1f-534ccb654cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:57.126607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-193f87a8-cfa6-4550-8a1f-534ccb654cbc None None] MFA Rules not processed for user `759f7e79147b420b9a757d8819432027`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:05:57.127884 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-193f87a8-cfa6-4550-8a1f-534ccb654cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:57.196961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1318/2606] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:05:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:05:57.207606 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad668b-f817-49f3-bb07-e75c69dd2ce1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:05:57.209327 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-24ad668b-f817-49f3-bb07-e75c69dd2ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:05:57.265867 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad668b-f817-49f3-bb07-e75c69dd2ce1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:05:57.269521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd04fd-7f57-4fa4-8072-c695dd00d5d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd59a351971e94aa38a22e8113207e5b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:05:57.270201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd04fd-7f57-4fa4-8072-c695dd00d5d8 service glance] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:05:57.270486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd04fd-7f57-4fa4-8072-c695dd00d5d8 service glance] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:05:57.270762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd04fd-7f57-4fa4-8072-c695dd00d5d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:05:57.279008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd04fd-7f57-4fa4-8072-c695dd00d5d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:05:57.280424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd04fd-7f57-4fa4-8072-c695dd00d5d8 service glance] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:05:57.289153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1289/2607] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:05:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:02.889429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f87a8-cfa6-4550-8a1f-534ccb654cbc None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:02.891084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-193f87a8-cfa6-4550-8a1f-534ccb654cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:03.093225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f87a8-cfa6-4550-8a1f-534ccb654cbc None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:03.097738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:03.098667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:03.098982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:03.099259 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] PATH_INFO: `/v3/users/39aa02a6acfb4ade9c6093908ec98084` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:03.100435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39aa02a6acfb4ade9c6093908ec98084)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:03.102390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:03.226459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:03.232211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39aa02a6acfb4ade9c6093908ec98084'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:03.232922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39aa02a6acfb4ade9c6093908ec98084'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:03.239625 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39aa02a6acfb4ade9c6093908ec98084'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:03.251283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39aa02a6acfb4ade9c6093908ec98084'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:03.304278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39aa02a6acfb4ade9c6093908ec98084'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:03.311695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1319/2608] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:02 2022] DELETE /identity/v3/users/39aa02a6acfb4ade9c6093908ec98084 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:03.322005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade61eb-7927-40ff-8484-6dcd1181916b None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:03.323074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade61eb-7927-40ff-8484-6dcd1181916b None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:03.323646 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade61eb-7927-40ff-8484-6dcd1181916b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:03.343350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1ade61eb-7927-40ff-8484-6dcd1181916b None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:03.344923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1ade61eb-7927-40ff-8484-6dcd1181916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:03.397709 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:03.410409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1290/2609] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:03.420994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:03.422528 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:03.482072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e96dd-5c52-4b67-ae01-de716d21ab80 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:03.485765 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe527c9-f28b-481e-bdff-2211797f0f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:03.486441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe527c9-f28b-481e-bdff-2211797f0f43 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:03.486722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe527c9-f28b-481e-bdff-2211797f0f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:03.487007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe527c9-f28b-481e-bdff-2211797f0f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:03.496958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe527c9-f28b-481e-bdff-2211797f0f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:03.498500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe527c9-f28b-481e-bdff-2211797f0f43 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:03.508562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1320/2610] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:03.892259 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade61eb-7927-40ff-8484-6dcd1181916b None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:03.893930 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1ade61eb-7927-40ff-8484-6dcd1181916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:03.919195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade61eb-7927-40ff-8484-6dcd1181916b None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:03.923859 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:03.924721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:03.925003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:03.925300 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] PATH_INFO: `/v3/projects/698b89f5dc604e568b1c133eda7b2804` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:03.926557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=698b89f5dc604e568b1c133eda7b2804)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:03.928482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:04.037307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '698b89f5dc604e568b1c133eda7b2804', 'request_id': 'req-8f110244-7ed6-40df-8b61-699605704c2d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.043310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '698b89f5dc604e568b1c133eda7b2804', 'request_id': 'req-8f110244-7ed6-40df-8b61-699605704c2d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.104240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '698b89f5dc604e568b1c133eda7b2804', 'request_id': 'req-8f110244-7ed6-40df-8b61-699605704c2d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.117998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] The token cache is being invalidate because project 698b89f5dc604e568b1c133eda7b2804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:04.118291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.118923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.120513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1291/2611] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:03 2022] DELETE /identity/v3/projects/698b89f5dc604e568b1c133eda7b2804 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:04.125212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe527c9-f28b-481e-bdff-2211797f0f43 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:04.135099 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe527c9-f28b-481e-bdff-2211797f0f43 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:04.194554 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:04.195430 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:04.195710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:04.195983 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] PATH_INFO: `/v3/users/12a69fc482a34a5fa7c34f9b646437a5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:04.197258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a69fc482a34a5fa7c34f9b646437a5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:04.199224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:04.370271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a69fc482a34a5fa7c34f9b646437a5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.370929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a69fc482a34a5fa7c34f9b646437a5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.377859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a69fc482a34a5fa7c34f9b646437a5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.382644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:04.391017 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a69fc482a34a5fa7c34f9b646437a5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.464483 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ab7b7a98-8ea2-45f3-8634-88976a735e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a69fc482a34a5fa7c34f9b646437a5'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:04.471628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1321/2612] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:04 2022] DELETE /identity/v3/users/12a69fc482a34a5fa7c34f9b646437a5 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:04.883416 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:04.884948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:04.899127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f110244-7ed6-40df-8b61-699605704c2d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:04.956971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:04.957865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:04.958180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:04.958460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] PATH_INFO: `/v3/projects/d1f4e80590b94eeda8f096e635d845c3` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:04.959627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f4e80590b94eeda8f096e635d845c3)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:04.961602 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:05.035472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f4e80590b94eeda8f096e635d845c3', 'request_id': 'req-5826b281-ba75-45d9-a4bd-8b382305bb0a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:05.041151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f4e80590b94eeda8f096e635d845c3', 'request_id': 'req-5826b281-ba75-45d9-a4bd-8b382305bb0a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:05.081473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f4e80590b94eeda8f096e635d845c3', 'request_id': 'req-5826b281-ba75-45d9-a4bd-8b382305bb0a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:05.094088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] The token cache is being invalidate because project d1f4e80590b94eeda8f096e635d845c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:05.094375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:05.095041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5826b281-ba75-45d9-a4bd-8b382305bb0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:05.096568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1292/2613] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:04 2022] DELETE /identity/v3/projects/d1f4e80590b94eeda8f096e635d845c3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:05.101550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80917be-f50a-45ad-94e6-9bc797e03d32 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.101841 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80917be-f50a-45ad-94e6-9bc797e03d32 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.102155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80917be-f50a-45ad-94e6-9bc797e03d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.119782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-d80917be-f50a-45ad-94e6-9bc797e03d32 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:05.120978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d80917be-f50a-45ad-94e6-9bc797e03d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:05.179229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1322/2614] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:06:05.186965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae06fc-e922-4d74-8f87-318526d55ab4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.187255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae06fc-e922-4d74-8f87-318526d55ab4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.187530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae06fc-e922-4d74-8f87-318526d55ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.200075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:05.206194 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-5cae06fc-e922-4d74-8f87-318526d55ab4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:05.214652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-5cae06fc-e922-4d74-8f87-318526d55ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:05.215820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1293/2615] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:05.219904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80917be-f50a-45ad-94e6-9bc797e03d32 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:05.229508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80917be-f50a-45ad-94e6-9bc797e03d32 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:05.234243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6d06c-457f-4329-833d-bca16b1690df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:05.234966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6d06c-457f-4329-833d-bca16b1690df admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.235245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6d06c-457f-4329-833d-bca16b1690df admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.235522 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6d06c-457f-4329-833d-bca16b1690df admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.236233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6d06c-457f-4329-833d-bca16b1690df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:06:05.236535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6d06c-457f-4329-833d-bca16b1690df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:05.238145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6d06c-457f-4329-833d-bca16b1690df admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:05.249397 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1323/2616] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:06:05.253697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae06fc-e922-4d74-8f87-318526d55ab4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:05.263460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae06fc-e922-4d74-8f87-318526d55ab4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:05.268732 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc72b10-abe5-41e5-a887-8cae129eb99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:05.271981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc72b10-abe5-41e5-a887-8cae129eb99d admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.271981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc72b10-abe5-41e5-a887-8cae129eb99d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.271981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc72b10-abe5-41e5-a887-8cae129eb99d admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.271981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc72b10-abe5-41e5-a887-8cae129eb99d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:05.272631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc72b10-abe5-41e5-a887-8cae129eb99d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:05.389545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1294/2617] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:06:05 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:05.391220 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6d06c-457f-4329-833d-bca16b1690df admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:05.401153 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6d06c-457f-4329-833d-bca16b1690df admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:05.458860 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a137f31-f4e2-4007-a7a2-61fda3adf699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:05.459441 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a137f31-f4e2-4007-a7a2-61fda3adf699 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.459722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a137f31-f4e2-4007-a7a2-61fda3adf699 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.459996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a137f31-f4e2-4007-a7a2-61fda3adf699 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.460698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a137f31-f4e2-4007-a7a2-61fda3adf699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:05.462643 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a137f31-f4e2-4007-a7a2-61fda3adf699 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:05.528055 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1324/2618] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:06:05 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:05.532477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc72b10-abe5-41e5-a887-8cae129eb99d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:05.543034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc72b10-abe5-41e5-a887-8cae129eb99d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:05.548031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab5598-5f4b-4411-892e-6ffa565f16ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:05.548791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab5598-5f4b-4411-892e-6ffa565f16ba admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.549149 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab5598-5f4b-4411-892e-6ffa565f16ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.549569 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab5598-5f4b-4411-892e-6ffa565f16ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.550392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab5598-5f4b-4411-892e-6ffa565f16ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:05.552052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab5598-5f4b-4411-892e-6ffa565f16ba admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:05.563499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1295/2619] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:05.567345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a137f31-f4e2-4007-a7a2-61fda3adf699 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:05.578186 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a137f31-f4e2-4007-a7a2-61fda3adf699 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:05.582713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a0d0d-48ef-4903-a98a-e9770baedf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:05.583391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a0d0d-48ef-4903-a98a-e9770baedf20 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.583673 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a0d0d-48ef-4903-a98a-e9770baedf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.583947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a0d0d-48ef-4903-a98a-e9770baedf20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.584663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a0d0d-48ef-4903-a98a-e9770baedf20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:05.586281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a0d0d-48ef-4903-a98a-e9770baedf20 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:05.596758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1325/2620] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:05.600956 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab5598-5f4b-4411-892e-6ffa565f16ba admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:05.610673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab5598-5f4b-4411-892e-6ffa565f16ba admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:05.615246 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c162a7-cced-4aa0-be90-d03da5875087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:05.616120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c162a7-cced-4aa0-be90-d03da5875087 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.616402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c162a7-cced-4aa0-be90-d03da5875087 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.616679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c162a7-cced-4aa0-be90-d03da5875087 admin admin] PATH_INFO: `/v3/projects/09793cf6f2a94b26866f50f79dd757ce/users/5ff1a1430b3949d78766d6781dadac71/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.632592 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c162a7-cced-4aa0-be90-d03da5875087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09793cf6f2a94b26866f50f79dd757ce, user_id=5ff1a1430b3949d78766d6781dadac71, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:05.635337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c162a7-cced-4aa0-be90-d03da5875087 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:05.699431 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1296/2621] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:05 2022] PUT /identity/v3/projects/09793cf6f2a94b26866f50f79dd757ce/users/5ff1a1430b3949d78766d6781dadac71/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:06:05.712850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec224a4-83b4-44b2-b349-869830d29aec None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.713148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec224a4-83b4-44b2-b349-869830d29aec None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.713500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec224a4-83b4-44b2-b349-869830d29aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.741698 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9ec224a4-83b4-44b2-b349-869830d29aec None None] MFA Rules not processed for user `5ff1a1430b3949d78766d6781dadac71`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:05.742972 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9ec224a4-83b4-44b2-b349-869830d29aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:05.776311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:05.807950 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1326/2622] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:06:05.813189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c162a7-cced-4aa0-be90-d03da5875087 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:05.875774 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c162a7-cced-4aa0-be90-d03da5875087 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:05.880476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c670b-e980-458d-a743-b8ed42915814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:05.881122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c670b-e980-458d-a743-b8ed42915814 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:05.881412 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c670b-e980-458d-a743-b8ed42915814 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:05.881985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c670b-e980-458d-a743-b8ed42915814 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:05.882683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c670b-e980-458d-a743-b8ed42915814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:05.884588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c670b-e980-458d-a743-b8ed42915814 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:05.981590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1297/2623] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:06:05 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:05.985862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec224a4-83b4-44b2-b349-869830d29aec None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:05.998838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec224a4-83b4-44b2-b349-869830d29aec None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:06.054307 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afc336-600e-4c34-91dc-87d56e036cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:06.054971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afc336-600e-4c34-91dc-87d56e036cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:06.055256 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afc336-600e-4c34-91dc-87d56e036cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:06.055535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afc336-600e-4c34-91dc-87d56e036cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:06.056229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97afc336-600e-4c34-91dc-87d56e036cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:06.058197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97afc336-600e-4c34-91dc-87d56e036cc3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:06.139737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1327/2624] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:06:05 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:06.143968 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c670b-e980-458d-a743-b8ed42915814 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:06.154969 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c670b-e980-458d-a743-b8ed42915814 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:06.159652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d81edb-86c2-4add-9029-ad7513c983ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:06.160466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d81edb-86c2-4add-9029-ad7513c983ee admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:06.160759 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d81edb-86c2-4add-9029-ad7513c983ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:06.161044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d81edb-86c2-4add-9029-ad7513c983ee admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:06.161797 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d81edb-86c2-4add-9029-ad7513c983ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:06.163500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d81edb-86c2-4add-9029-ad7513c983ee admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:06.174619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1298/2625] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:06.178870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afc336-600e-4c34-91dc-87d56e036cc3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:06.192169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afc336-600e-4c34-91dc-87d56e036cc3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:06.200785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49753bf4-79b2-46d2-ae9f-c86538802f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:06.201744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49753bf4-79b2-46d2-ae9f-c86538802f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:06.202073 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49753bf4-79b2-46d2-ae9f-c86538802f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:06.202338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49753bf4-79b2-46d2-ae9f-c86538802f0b admin admin] PATH_INFO: `/v3/projects/fa088fb1d9fe4605847ef0a84b8eb54a/users/24adf6609e6a48ab874dfa2d12b385bc/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:06.219267 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49753bf4-79b2-46d2-ae9f-c86538802f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa088fb1d9fe4605847ef0a84b8eb54a, user_id=24adf6609e6a48ab874dfa2d12b385bc, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:06.221962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49753bf4-79b2-46d2-ae9f-c86538802f0b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:06.311822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1328/2626] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:06 2022] PUT /identity/v3/projects/fa088fb1d9fe4605847ef0a84b8eb54a/users/24adf6609e6a48ab874dfa2d12b385bc/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:06.315714 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d81edb-86c2-4add-9029-ad7513c983ee admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:06.327255 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d81edb-86c2-4add-9029-ad7513c983ee admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:06.395906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed76c5-dec7-449b-964a-14da32150363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:06.396620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed76c5-dec7-449b-964a-14da32150363 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:06.396906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed76c5-dec7-449b-964a-14da32150363 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:06.397186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed76c5-dec7-449b-964a-14da32150363 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:06.398063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed76c5-dec7-449b-964a-14da32150363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:06.399731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed76c5-dec7-449b-964a-14da32150363 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:06.415744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1299/2627] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:06.423038 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49753bf4-79b2-46d2-ae9f-c86538802f0b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:06.433922 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49753bf4-79b2-46d2-ae9f-c86538802f0b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:06.439210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db7e16-00c0-4260-ac9f-125e22c58943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:06.440155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7e16-00c0-4260-ac9f-125e22c58943 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:06.440454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7e16-00c0-4260-ac9f-125e22c58943 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:06.440729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7e16-00c0-4260-ac9f-125e22c58943 admin admin] PATH_INFO: `/v3/projects/fa088fb1d9fe4605847ef0a84b8eb54a/users/24adf6609e6a48ab874dfa2d12b385bc/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:06.442892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db7e16-00c0-4260-ac9f-125e22c58943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa088fb1d9fe4605847ef0a84b8eb54a, user_id=24adf6609e6a48ab874dfa2d12b385bc, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:06.446247 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db7e16-00c0-4260-ac9f-125e22c58943 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:06.458693 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1329/2628] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:06 2022] PUT /identity/v3/projects/fa088fb1d9fe4605847ef0a84b8eb54a/users/24adf6609e6a48ab874dfa2d12b385bc/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:06.525623 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1f2a4-89d4-4f8d-8bab-9c9c34d6600d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:06.526145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1f2a4-89d4-4f8d-8bab-9c9c34d6600d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:06.530153 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1f2a4-89d4-4f8d-8bab-9c9c34d6600d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:06.553535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:06.556849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-daf1f2a4-89d4-4f8d-8bab-9c9c34d6600d None None] MFA Rules not processed for user `24adf6609e6a48ab874dfa2d12b385bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:06.558183 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-daf1f2a4-89d4-4f8d-8bab-9c9c34d6600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:06.646733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1300/2629] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:06.658078 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db7e16-00c0-4260-ac9f-125e22c58943 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:06.659845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d6db7e16-00c0-4260-ac9f-125e22c58943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:06.720536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db7e16-00c0-4260-ac9f-125e22c58943 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:06.724168 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f4fc4-c43b-45fa-9079-f95a2d80131e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:06.724826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f4fc4-c43b-45fa-9079-f95a2d80131e service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:06.725110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f4fc4-c43b-45fa-9079-f95a2d80131e service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:06.725393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f4fc4-c43b-45fa-9079-f95a2d80131e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:06.736179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f4fc4-c43b-45fa-9079-f95a2d80131e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:06.737793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f4fc4-c43b-45fa-9079-f95a2d80131e service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:06.749865 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1330/2630] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:07.350220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1f2a4-89d4-4f8d-8bab-9c9c34d6600d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:07.418143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1f2a4-89d4-4f8d-8bab-9c9c34d6600d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:07.424521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:07.425393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:07.425666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:07.425969 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] PATH_INFO: `/v3/users/5ff1a1430b3949d78766d6781dadac71` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:07.427617 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff1a1430b3949d78766d6781dadac71)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:07.429845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:07.630592 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff1a1430b3949d78766d6781dadac71'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:07.631207 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff1a1430b3949d78766d6781dadac71'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:07.645198 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff1a1430b3949d78766d6781dadac71'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:07.652457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff1a1430b3949d78766d6781dadac71'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:07.700011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff1a1430b3949d78766d6781dadac71'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:07.713979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1301/2631] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:07 2022] DELETE /identity/v3/users/5ff1a1430b3949d78766d6781dadac71 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:07.717327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd86f6-a2ed-4f78-a050-24f034bc9824 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:07.718446 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd86f6-a2ed-4f78-a050-24f034bc9824 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:07.718832 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd86f6-a2ed-4f78-a050-24f034bc9824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:07.736393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6abd86f6-a2ed-4f78-a050-24f034bc9824 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:07.737626 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6abd86f6-a2ed-4f78-a050-24f034bc9824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:07.765068 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:07.803027 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1331/2632] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:07.815519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:07.820521 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:07.856710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:07.909570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c854918-1027-429a-938a-ea88ad9ca846 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:07.913289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebab419-c212-4b0b-969c-6e04229855d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:07.914214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebab419-c212-4b0b-969c-6e04229855d0 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:07.914427 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebab419-c212-4b0b-969c-6e04229855d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:07.914934 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebab419-c212-4b0b-969c-6e04229855d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:07.926979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebab419-c212-4b0b-969c-6e04229855d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:07.928754 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebab419-c212-4b0b-969c-6e04229855d0 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:07.941193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1302/2633] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:08.369851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd86f6-a2ed-4f78-a050-24f034bc9824 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:08.371676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6abd86f6-a2ed-4f78-a050-24f034bc9824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:08.385497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd86f6-a2ed-4f78-a050-24f034bc9824 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:08.390699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:08.391592 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:08.391877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:08.392203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] PATH_INFO: `/v3/projects/09793cf6f2a94b26866f50f79dd757ce` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:08.393700 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09793cf6f2a94b26866f50f79dd757ce)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:08.395738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:08.504237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09793cf6f2a94b26866f50f79dd757ce', 'request_id': 'req-f87304c8-aa22-49c6-8c9c-ec04f9023af3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:08.510100 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09793cf6f2a94b26866f50f79dd757ce', 'request_id': 'req-f87304c8-aa22-49c6-8c9c-ec04f9023af3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:08.590194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09793cf6f2a94b26866f50f79dd757ce', 'request_id': 'req-f87304c8-aa22-49c6-8c9c-ec04f9023af3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:08.601533 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] The token cache is being invalidate because project 09793cf6f2a94b26866f50f79dd757ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:08.601822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:08.602555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:08.604124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1332/2634] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:08 2022] DELETE /identity/v3/projects/09793cf6f2a94b26866f50f79dd757ce => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:08.608884 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebab419-c212-4b0b-969c-6e04229855d0 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:08.618487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebab419-c212-4b0b-969c-6e04229855d0 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:08.685169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:08.686336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:08.687065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:08.687545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] PATH_INFO: `/v3/users/24adf6609e6a48ab874dfa2d12b385bc` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:08.688881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24adf6609e6a48ab874dfa2d12b385bc)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:08.690932 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:08.936258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24adf6609e6a48ab874dfa2d12b385bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:08.936930 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24adf6609e6a48ab874dfa2d12b385bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:08.953241 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24adf6609e6a48ab874dfa2d12b385bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:08.960432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24adf6609e6a48ab874dfa2d12b385bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:09.064125 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-00c5bd3c-d77f-4c79-8bbe-5ed9e994ba59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24adf6609e6a48ab874dfa2d12b385bc'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:09.070532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1303/2635] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:08 2022] DELETE /identity/v3/users/24adf6609e6a48ab874dfa2d12b385bc => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:09.505794 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:09.507850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:09.522291 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87304c8-aa22-49c6-8c9c-ec04f9023af3 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:09.591009 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:09.591870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:09.592149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:09.592422 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] PATH_INFO: `/v3/projects/fa088fb1d9fe4605847ef0a84b8eb54a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:09.594630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa088fb1d9fe4605847ef0a84b8eb54a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:09.596561 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:09.726274 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa088fb1d9fe4605847ef0a84b8eb54a', 'request_id': 'req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:09.731116 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:09.733317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa088fb1d9fe4605847ef0a84b8eb54a', 'request_id': 'req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:09.808196 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa088fb1d9fe4605847ef0a84b8eb54a', 'request_id': 'req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:09.820638 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] The token cache is being invalidate because project fa088fb1d9fe4605847ef0a84b8eb54a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:09.821026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:09.821720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3fcea8a8-5414-488e-8ce1-06e1efcb3fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:09.823402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1333/2636] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:09 2022] DELETE /identity/v3/projects/fa088fb1d9fe4605847ef0a84b8eb54a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:09.828491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563023d-8738-4a30-adb0-0a233a359ba4 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:09.829040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563023d-8738-4a30-adb0-0a233a359ba4 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:09.829192 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563023d-8738-4a30-adb0-0a233a359ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:09.848651 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0563023d-8738-4a30-adb0-0a233a359ba4 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:09.849892 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0563023d-8738-4a30-adb0-0a233a359ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:09.890791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:09.923632 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1304/2637] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:09.931737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787803d-12ba-4d2c-98f0-ec646ccc7dbe None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:09.932043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787803d-12ba-4d2c-98f0-ec646ccc7dbe None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:09.932312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787803d-12ba-4d2c-98f0-ec646ccc7dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:09.953350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-1787803d-12ba-4d2c-98f0-ec646ccc7dbe None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:09.954687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1787803d-12ba-4d2c-98f0-ec646ccc7dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:09.963683 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1334/2638] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:09.968568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563023d-8738-4a30-adb0-0a233a359ba4 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:09.978854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563023d-8738-4a30-adb0-0a233a359ba4 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:09.984583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc6fd6-2022-4679-8c14-dbb6f9bbb262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:09.985328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc6fd6-2022-4679-8c14-dbb6f9bbb262 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:09.985679 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc6fd6-2022-4679-8c14-dbb6f9bbb262 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:09.986082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc6fd6-2022-4679-8c14-dbb6f9bbb262 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:09.986895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc6fd6-2022-4679-8c14-dbb6f9bbb262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:06:09.987266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc6fd6-2022-4679-8c14-dbb6f9bbb262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:09.989018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc6fd6-2022-4679-8c14-dbb6f9bbb262 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:10.001661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1305/2639] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:06:10.006288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787803d-12ba-4d2c-98f0-ec646ccc7dbe None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:10.016493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787803d-12ba-4d2c-98f0-ec646ccc7dbe None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:10.021575 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10895708-48ad-41b9-8737-506f645b8aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:10.022408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10895708-48ad-41b9-8737-506f645b8aab admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:10.028006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10895708-48ad-41b9-8737-506f645b8aab admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:10.028006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10895708-48ad-41b9-8737-506f645b8aab admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:10.028006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10895708-48ad-41b9-8737-506f645b8aab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:10.028006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10895708-48ad-41b9-8737-506f645b8aab admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:10.112244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1335/2640] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:06:10 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:06:10.116414 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc6fd6-2022-4679-8c14-dbb6f9bbb262 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:10.127206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc6fd6-2022-4679-8c14-dbb6f9bbb262 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:10.195847 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa8738-1cf0-4589-865f-726427526aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:10.196511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa8738-1cf0-4589-865f-726427526aeb admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:10.196794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa8738-1cf0-4589-865f-726427526aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:10.197071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa8738-1cf0-4589-865f-726427526aeb admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:10.197805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa8738-1cf0-4589-865f-726427526aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:10.200099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa8738-1cf0-4589-865f-726427526aeb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:10.293965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1306/2641] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:06:10 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:06:10.300824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10895708-48ad-41b9-8737-506f645b8aab admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:10.312141 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10895708-48ad-41b9-8737-506f645b8aab admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:10.316845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac660b-f139-4266-8da1-2286198fd006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:10.317504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac660b-f139-4266-8da1-2286198fd006 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:10.317790 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac660b-f139-4266-8da1-2286198fd006 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:10.318114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac660b-f139-4266-8da1-2286198fd006 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:10.318846 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac660b-f139-4266-8da1-2286198fd006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:10.320512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac660b-f139-4266-8da1-2286198fd006 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:10.330930 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1336/2642] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:06:10.334951 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa8738-1cf0-4589-865f-726427526aeb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:10.345282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa8738-1cf0-4589-865f-726427526aeb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:10.349945 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d03a4-372d-4f62-9bd2-a26c3d9dcc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:10.350595 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d03a4-372d-4f62-9bd2-a26c3d9dcc34 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:10.354127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d03a4-372d-4f62-9bd2-a26c3d9dcc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:10.354127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d03a4-372d-4f62-9bd2-a26c3d9dcc34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:10.354127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d03a4-372d-4f62-9bd2-a26c3d9dcc34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:10.354127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d03a4-372d-4f62-9bd2-a26c3d9dcc34 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:10.364118 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1307/2643] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:06:10.368155 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac660b-f139-4266-8da1-2286198fd006 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:10.378642 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac660b-f139-4266-8da1-2286198fd006 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:10.383571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4751477-8ee4-4310-b674-2b27ea291d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:10.384443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4751477-8ee4-4310-b674-2b27ea291d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:10.384722 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4751477-8ee4-4310-b674-2b27ea291d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:10.385177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4751477-8ee4-4310-b674-2b27ea291d69 admin admin] PATH_INFO: `/v3/projects/6681180051264d52a8fb7b37fabfde87/users/b0a9608f59414e8b849774df9cd52a0c/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:10.400892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4751477-8ee4-4310-b674-2b27ea291d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6681180051264d52a8fb7b37fabfde87, user_id=b0a9608f59414e8b849774df9cd52a0c, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:10.403543 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4751477-8ee4-4310-b674-2b27ea291d69 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:10.476248 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1337/2644] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:10 2022] PUT /identity/v3/projects/6681180051264d52a8fb7b37fabfde87/users/b0a9608f59414e8b849774df9cd52a0c/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:10.488468 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539ae06-5bb8-4c23-9684-9f59f9c62a61 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:10.488764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539ae06-5bb8-4c23-9684-9f59f9c62a61 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:10.489051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539ae06-5bb8-4c23-9684-9f59f9c62a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:10.516726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1539ae06-5bb8-4c23-9684-9f59f9c62a61 None None] MFA Rules not processed for user `b0a9608f59414e8b849774df9cd52a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:10.517949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1539ae06-5bb8-4c23-9684-9f59f9c62a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:10.584857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1308/2645] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:06:10.597994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4751477-8ee4-4310-b674-2b27ea291d69 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:10.598677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b4751477-8ee4-4310-b674-2b27ea291d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:10.657745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4751477-8ee4-4310-b674-2b27ea291d69 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:10.661461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b26a5-6d61-4e72-9bc1-57f20a8c4b7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:10.662134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b26a5-6d61-4e72-9bc1-57f20a8c4b7d service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:10.662421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b26a5-6d61-4e72-9bc1-57f20a8c4b7d service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:10.662699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b26a5-6d61-4e72-9bc1-57f20a8c4b7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:10.671842 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b26a5-6d61-4e72-9bc1-57f20a8c4b7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:10.673265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b26a5-6d61-4e72-9bc1-57f20a8c4b7d service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:10.682049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1338/2646] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:12.729725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093936aa-3a3f-44b4-9789-8f34c7f1aa10 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:12.730783 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093936aa-3a3f-44b4-9789-8f34c7f1aa10 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:12.738547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093936aa-3a3f-44b4-9789-8f34c7f1aa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:12.751140 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-093936aa-3a3f-44b4-9789-8f34c7f1aa10 None None] MFA Rules not processed for user `ce77221a2c8d4c4395bed092c1967b28`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:12.752364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-093936aa-3a3f-44b4-9789-8f34c7f1aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:12.781809 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:12.806422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1309/2647] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:20.678174 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=2,keystone:SELECT=58,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:21.952065 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b26a5-6d61-4e72-9bc1-57f20a8c4b7d service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:21.953604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e33b26a5-6d61-4e72-9bc1-57f20a8c4b7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:22.020403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b26a5-6d61-4e72-9bc1-57f20a8c4b7d service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:22.024874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:22.025747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:22.026058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:22.026332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] PATH_INFO: `/v3/users/759f7e79147b420b9a757d8819432027` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:22.027521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=759f7e79147b420b9a757d8819432027)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:22.030054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:22.244451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '759f7e79147b420b9a757d8819432027'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:22.245119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '759f7e79147b420b9a757d8819432027'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:22.252145 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '759f7e79147b420b9a757d8819432027'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:22.264271 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '759f7e79147b420b9a757d8819432027'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:22.349931 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '759f7e79147b420b9a757d8819432027'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:22.357749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1339/2648] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:21 2022] DELETE /identity/v3/users/759f7e79147b420b9a757d8819432027 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:22.362936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee5bcb-8b58-4233-ab77-483628b86580 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:22.363434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee5bcb-8b58-4233-ab77-483628b86580 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:22.363536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee5bcb-8b58-4233-ab77-483628b86580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:22.380570 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-43ee5bcb-8b58-4233-ab77-483628b86580 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:22.381961 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-43ee5bcb-8b58-4233-ab77-483628b86580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:22.440048 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1310/2649] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:22.449651 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:22.451283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:22.507678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440a75f-3524-4fbb-988e-ec40715d5103 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:22.511265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e6129-ced4-4b6f-b5e2-6ae0b8480282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:22.511899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e6129-ced4-4b6f-b5e2-6ae0b8480282 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:22.512184 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e6129-ced4-4b6f-b5e2-6ae0b8480282 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:22.512461 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e6129-ced4-4b6f-b5e2-6ae0b8480282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:22.521864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e6129-ced4-4b6f-b5e2-6ae0b8480282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:22.523381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e6129-ced4-4b6f-b5e2-6ae0b8480282 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:22.534023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1340/2650] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:22.974466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee5bcb-8b58-4233-ab77-483628b86580 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:22.975897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-43ee5bcb-8b58-4233-ab77-483628b86580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:22.987974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee5bcb-8b58-4233-ab77-483628b86580 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:22.992739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:22.993608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:22.993973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:22.994193 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] PATH_INFO: `/v3/projects/6a5a45705c3c4280bd89956c75e03ee1` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:23.001220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5a45705c3c4280bd89956c75e03ee1)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:23.003248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:23.153390 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5a45705c3c4280bd89956c75e03ee1', 'request_id': 'req-55a05f10-c74e-44e2-afed-e21afde59100'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:23.159338 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5a45705c3c4280bd89956c75e03ee1', 'request_id': 'req-55a05f10-c74e-44e2-afed-e21afde59100'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:23.281918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5a45705c3c4280bd89956c75e03ee1', 'request_id': 'req-55a05f10-c74e-44e2-afed-e21afde59100'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:23.293384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] The token cache is being invalidate because project 6a5a45705c3c4280bd89956c75e03ee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:23.293672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:23.294320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-55a05f10-c74e-44e2-afed-e21afde59100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:23.295862 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1311/2651] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:22 2022] DELETE /identity/v3/projects/6a5a45705c3c4280bd89956c75e03ee1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:23.300721 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47691c06-c710-4536-aced-3404d330c380 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:23.301176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47691c06-c710-4536-aced-3404d330c380 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:23.301309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47691c06-c710-4536-aced-3404d330c380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:23.321954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-47691c06-c710-4536-aced-3404d330c380 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:23.323173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-47691c06-c710-4536-aced-3404d330c380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:23.342024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:23.383979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1341/2652] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:23.392261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47594d36-896b-4bd7-a473-f45954890c03 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:23.392770 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47594d36-896b-4bd7-a473-f45954890c03 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:23.392889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47594d36-896b-4bd7-a473-f45954890c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:23.411323 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-47594d36-896b-4bd7-a473-f45954890c03 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:23.412534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-47594d36-896b-4bd7-a473-f45954890c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:23.420635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1312/2653] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:23.424919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47691c06-c710-4536-aced-3404d330c380 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:23.434830 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47691c06-c710-4536-aced-3404d330c380 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:23.439772 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe050-c24a-4c95-8e39-cdc3432eab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:23.440524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe050-c24a-4c95-8e39-cdc3432eab41 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:23.440814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe050-c24a-4c95-8e39-cdc3432eab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:23.441092 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe050-c24a-4c95-8e39-cdc3432eab41 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:23.441814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe050-c24a-4c95-8e39-cdc3432eab41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:06:23.442150 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe050-c24a-4c95-8e39-cdc3432eab41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:23.443793 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe050-c24a-4c95-8e39-cdc3432eab41 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:23.457785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1342/2654] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:06:23.460742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47594d36-896b-4bd7-a473-f45954890c03 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:23.470775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47594d36-896b-4bd7-a473-f45954890c03 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:23.475566 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97be16b-9d55-47f8-9c1f-ea0a6783b40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:23.476217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97be16b-9d55-47f8-9c1f-ea0a6783b40a admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:23.476524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97be16b-9d55-47f8-9c1f-ea0a6783b40a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:23.476827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97be16b-9d55-47f8-9c1f-ea0a6783b40a admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:23.477553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97be16b-9d55-47f8-9c1f-ea0a6783b40a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:23.479672 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97be16b-9d55-47f8-9c1f-ea0a6783b40a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:23.631779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1313/2655] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:06:23 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:23.635819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe050-c24a-4c95-8e39-cdc3432eab41 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:23.646879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe050-c24a-4c95-8e39-cdc3432eab41 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:23.704279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c08fe-db69-4e52-959e-b4e693eb72e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:23.704953 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c08fe-db69-4e52-959e-b4e693eb72e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:23.705243 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c08fe-db69-4e52-959e-b4e693eb72e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:23.705524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c08fe-db69-4e52-959e-b4e693eb72e7 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:23.706270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c08fe-db69-4e52-959e-b4e693eb72e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:23.708266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c08fe-db69-4e52-959e-b4e693eb72e7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:23.853571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1343/2656] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:06:23 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:23.857493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97be16b-9d55-47f8-9c1f-ea0a6783b40a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:23.869856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97be16b-9d55-47f8-9c1f-ea0a6783b40a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:23.873677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620889f5-af8c-4100-ac75-0a13b84b2fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:23.874467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620889f5-af8c-4100-ac75-0a13b84b2fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:23.874749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620889f5-af8c-4100-ac75-0a13b84b2fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:23.878406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620889f5-af8c-4100-ac75-0a13b84b2fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:23.878406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620889f5-af8c-4100-ac75-0a13b84b2fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:23.878406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620889f5-af8c-4100-ac75-0a13b84b2fd6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:23.888641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1314/2657] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:23.895792 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c08fe-db69-4e52-959e-b4e693eb72e7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:23.907699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c08fe-db69-4e52-959e-b4e693eb72e7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:23.912959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c033764-915a-41df-b014-2bc601f5bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:23.914118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c033764-915a-41df-b014-2bc601f5bdfa admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:23.914118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c033764-915a-41df-b014-2bc601f5bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:23.914317 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c033764-915a-41df-b014-2bc601f5bdfa admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:23.914942 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c033764-915a-41df-b014-2bc601f5bdfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:23.916550 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c033764-915a-41df-b014-2bc601f5bdfa admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:23.927589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1344/2658] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:23.932175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620889f5-af8c-4100-ac75-0a13b84b2fd6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:23.944444 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620889f5-af8c-4100-ac75-0a13b84b2fd6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:23.949230 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc68c-e299-4f52-8b45-73fcb7a7816d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:23.950428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc68c-e299-4f52-8b45-73fcb7a7816d admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:23.950712 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc68c-e299-4f52-8b45-73fcb7a7816d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:23.950990 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc68c-e299-4f52-8b45-73fcb7a7816d admin admin] PATH_INFO: `/v3/projects/ad786d15a1c74e89aa73cb2379cd783a/users/01402a48b47a4e25b587387dd18af04f/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:23.973673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cc68c-e299-4f52-8b45-73fcb7a7816d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad786d15a1c74e89aa73cb2379cd783a, user_id=01402a48b47a4e25b587387dd18af04f, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:23.982742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cc68c-e299-4f52-8b45-73fcb7a7816d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:24.077261 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:24.086464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1315/2659] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:23 2022] PUT /identity/v3/projects/ad786d15a1c74e89aa73cb2379cd783a/users/01402a48b47a4e25b587387dd18af04f/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:24.097712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e0aa8-153c-4fe2-a3a5-1b97c296c00c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:24.098036 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e0aa8-153c-4fe2-a3a5-1b97c296c00c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:24.098326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e0aa8-153c-4fe2-a3a5-1b97c296c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:24.125044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2b3e0aa8-153c-4fe2-a3a5-1b97c296c00c None None] MFA Rules not processed for user `01402a48b47a4e25b587387dd18af04f`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:24.126355 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2b3e0aa8-153c-4fe2-a3a5-1b97c296c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:24.197978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1345/2660] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:24.203065 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc68c-e299-4f52-8b45-73fcb7a7816d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:24.288175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc68c-e299-4f52-8b45-73fcb7a7816d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:24.294085 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a7afe-54cb-49c4-a69f-7c8713d99320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:24.294779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a7afe-54cb-49c4-a69f-7c8713d99320 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:24.295044 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a7afe-54cb-49c4-a69f-7c8713d99320 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:24.295359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a7afe-54cb-49c4-a69f-7c8713d99320 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:24.296059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a7afe-54cb-49c4-a69f-7c8713d99320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:24.298364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a7afe-54cb-49c4-a69f-7c8713d99320 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:24.432107 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1316/2661] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:06:24 2022] POST /identity/v3/projects => generated 377 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:24.437404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e0aa8-153c-4fe2-a3a5-1b97c296c00c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:24.448529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e0aa8-153c-4fe2-a3a5-1b97c296c00c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:24.461459 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:24.503273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b6b17-c5bf-452f-8e83-0736fb7b3dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:24.503929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b6b17-c5bf-452f-8e83-0736fb7b3dbf admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:24.504211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b6b17-c5bf-452f-8e83-0736fb7b3dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:24.504486 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b6b17-c5bf-452f-8e83-0736fb7b3dbf admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:24.505177 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b6b17-c5bf-452f-8e83-0736fb7b3dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:24.507364 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b6b17-c5bf-452f-8e83-0736fb7b3dbf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:24.733044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1346/2662] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:06:24 2022] POST /identity/v3/users => generated 289 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:06:24.737718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a7afe-54cb-49c4-a69f-7c8713d99320 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:24.748277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a7afe-54cb-49c4-a69f-7c8713d99320 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:24.753737 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5daaf9-a92f-4682-85d7-18c3e2335256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:24.754500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5daaf9-a92f-4682-85d7-18c3e2335256 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:24.754788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5daaf9-a92f-4682-85d7-18c3e2335256 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:24.755066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5daaf9-a92f-4682-85d7-18c3e2335256 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:24.755787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5daaf9-a92f-4682-85d7-18c3e2335256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:24.757642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5daaf9-a92f-4682-85d7-18c3e2335256 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:24.768114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1317/2663] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:24.771917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b6b17-c5bf-452f-8e83-0736fb7b3dbf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:24.781749 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b6b17-c5bf-452f-8e83-0736fb7b3dbf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:24.786249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e409e6-fde3-4a43-bd82-095f0ba3395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:24.786894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e409e6-fde3-4a43-bd82-095f0ba3395f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:24.787173 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e409e6-fde3-4a43-bd82-095f0ba3395f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:24.787444 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e409e6-fde3-4a43-bd82-095f0ba3395f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:24.788154 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e409e6-fde3-4a43-bd82-095f0ba3395f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:24.789735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e409e6-fde3-4a43-bd82-095f0ba3395f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:24.799757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1347/2664] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:24.803689 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5daaf9-a92f-4682-85d7-18c3e2335256 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:24.814231 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5daaf9-a92f-4682-85d7-18c3e2335256 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:24.818807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c5af-a3af-4048-b248-fe235b180599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:24.819677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c5af-a3af-4048-b248-fe235b180599 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:24.819963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c5af-a3af-4048-b248-fe235b180599 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:24.820244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c5af-a3af-4048-b248-fe235b180599 admin admin] PATH_INFO: `/v3/projects/36437dba73cf49559ee32d948e480ade/users/f2bd4d7a2dd04165b6611c13bde6aec1/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:24.834641 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c5af-a3af-4048-b248-fe235b180599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36437dba73cf49559ee32d948e480ade, user_id=f2bd4d7a2dd04165b6611c13bde6aec1, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:24.840955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c5af-a3af-4048-b248-fe235b180599 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:24.972982 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1318/2665] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:24 2022] PUT /identity/v3/projects/36437dba73cf49559ee32d948e480ade/users/f2bd4d7a2dd04165b6611c13bde6aec1/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:24.984602 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa9665-2eb0-4553-95f1-66687d001d0f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:24.984908 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa9665-2eb0-4553-95f1-66687d001d0f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:24.985187 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa9665-2eb0-4553-95f1-66687d001d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:25.012302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e7aa9665-2eb0-4553-95f1-66687d001d0f None None] MFA Rules not processed for user `f2bd4d7a2dd04165b6611c13bde6aec1`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:25.013532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e7aa9665-2eb0-4553-95f1-66687d001d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:25.079109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1348/2666] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:25.083579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c5af-a3af-4048-b248-fe235b180599 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:25.160452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c5af-a3af-4048-b248-fe235b180599 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:25.165499 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8de1b-d2ae-4627-bfb2-44dbd98889ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:25.166220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8de1b-d2ae-4627-bfb2-44dbd98889ab admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:25.166506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8de1b-d2ae-4627-bfb2-44dbd98889ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:25.166784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8de1b-d2ae-4627-bfb2-44dbd98889ab admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:25.167473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8de1b-d2ae-4627-bfb2-44dbd98889ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:25.169394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8de1b-d2ae-4627-bfb2-44dbd98889ab admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:25.295513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1319/2667] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:06:25 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:25.300118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa9665-2eb0-4553-95f1-66687d001d0f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:25.315937 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa9665-2eb0-4553-95f1-66687d001d0f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:25.380994 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3cf37-b684-4a81-885e-183a333a277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:25.381962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3cf37-b684-4a81-885e-183a333a277b admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:25.382252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3cf37-b684-4a81-885e-183a333a277b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:25.382535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3cf37-b684-4a81-885e-183a333a277b admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:25.383363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3cf37-b684-4a81-885e-183a333a277b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:25.385278 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3cf37-b684-4a81-885e-183a333a277b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:25.492996 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1349/2668] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:06:25 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:06:25.496886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8de1b-d2ae-4627-bfb2-44dbd98889ab admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:25.507973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8de1b-d2ae-4627-bfb2-44dbd98889ab admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:25.512682 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69c962-d045-413f-900d-ba40594835de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:25.513373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69c962-d045-413f-900d-ba40594835de admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:25.513659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69c962-d045-413f-900d-ba40594835de admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:25.514015 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69c962-d045-413f-900d-ba40594835de admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:25.522171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69c962-d045-413f-900d-ba40594835de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:25.522171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69c962-d045-413f-900d-ba40594835de admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:25.528541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1320/2669] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:25.533206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3cf37-b684-4a81-885e-183a333a277b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:25.543315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3cf37-b684-4a81-885e-183a333a277b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:25.548787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e921f4e-c975-4b24-a023-d151b135afa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:25.549648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e921f4e-c975-4b24-a023-d151b135afa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:25.549979 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e921f4e-c975-4b24-a023-d151b135afa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:25.550252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e921f4e-c975-4b24-a023-d151b135afa7 admin admin] PATH_INFO: `/v3/projects/703152913a4c427e8ae4416ff5c3583b/users/1aaccfe5e77843f89ab8dba35d1ca3be/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:25.565041 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e921f4e-c975-4b24-a023-d151b135afa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=703152913a4c427e8ae4416ff5c3583b, user_id=1aaccfe5e77843f89ab8dba35d1ca3be, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:25.567732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e921f4e-c975-4b24-a023-d151b135afa7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:25.681469 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1350/2670] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:25 2022] PUT /identity/v3/projects/703152913a4c427e8ae4416ff5c3583b/users/1aaccfe5e77843f89ab8dba35d1ca3be/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:25.685408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69c962-d045-413f-900d-ba40594835de admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:25.696204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69c962-d045-413f-900d-ba40594835de admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:25.755807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe15d0f-9f38-4b74-b9d9-907cfe4ce7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:25.756726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15d0f-9f38-4b74-b9d9-907cfe4ce7ed admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:25.757006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15d0f-9f38-4b74-b9d9-907cfe4ce7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:25.757278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15d0f-9f38-4b74-b9d9-907cfe4ce7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:25.758025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe15d0f-9f38-4b74-b9d9-907cfe4ce7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:25.759619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe15d0f-9f38-4b74-b9d9-907cfe4ce7ed admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:25.769239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1321/2671] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:25.773562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e921f4e-c975-4b24-a023-d151b135afa7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:25.783963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e921f4e-c975-4b24-a023-d151b135afa7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:25.788540 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b81cc7-26e1-4915-9572-e636b77ba8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:25.789431 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b81cc7-26e1-4915-9572-e636b77ba8d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:25.789712 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b81cc7-26e1-4915-9572-e636b77ba8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:25.790013 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b81cc7-26e1-4915-9572-e636b77ba8d1 admin admin] PATH_INFO: `/v3/projects/703152913a4c427e8ae4416ff5c3583b/users/1aaccfe5e77843f89ab8dba35d1ca3be/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:25.791976 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b81cc7-26e1-4915-9572-e636b77ba8d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=703152913a4c427e8ae4416ff5c3583b, user_id=1aaccfe5e77843f89ab8dba35d1ca3be, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:25.794593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b81cc7-26e1-4915-9572-e636b77ba8d1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:25.804737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1351/2672] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:25 2022] PUT /identity/v3/projects/703152913a4c427e8ae4416ff5c3583b/users/1aaccfe5e77843f89ab8dba35d1ca3be/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:25.815270 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5e5a1-dae6-4065-b302-72bcba4a7cca None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:25.815583 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5e5a1-dae6-4065-b302-72bcba4a7cca None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:25.815881 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5e5a1-dae6-4065-b302-72bcba4a7cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:25.841802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-e3e5e5a1-dae6-4065-b302-72bcba4a7cca None None] MFA Rules not processed for user `1aaccfe5e77843f89ab8dba35d1ca3be`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:25.843622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-e3e5e5a1-dae6-4065-b302-72bcba4a7cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:25.912655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1322/2673] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:25.925551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b81cc7-26e1-4915-9572-e636b77ba8d1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:25.929206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-77b81cc7-26e1-4915-9572-e636b77ba8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:25.987542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b81cc7-26e1-4915-9572-e636b77ba8d1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:25.991449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ff22c-88a7-4210-805c-56b0c0d9f38a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:25.992148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ff22c-88a7-4210-805c-56b0c0d9f38a service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:25.992508 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ff22c-88a7-4210-805c-56b0c0d9f38a service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:25.992782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ff22c-88a7-4210-805c-56b0c0d9f38a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:26.049004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ff22c-88a7-4210-805c-56b0c0d9f38a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:26.050553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ff22c-88a7-4210-805c-56b0c0d9f38a service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:26.060012 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1352/2674] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:06:26.139109 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:26.287607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:30.448067 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5e5a1-dae6-4065-b302-72bcba4a7cca None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:30.458771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5e5a1-dae6-4065-b302-72bcba4a7cca None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:30.462508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc831f6-c508-4a1e-a7fc-96d1ca556490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:30.463151 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc831f6-c508-4a1e-a7fc-96d1ca556490 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:30.463434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc831f6-c508-4a1e-a7fc-96d1ca556490 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:30.463708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc831f6-c508-4a1e-a7fc-96d1ca556490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:30.520408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc831f6-c508-4a1e-a7fc-96d1ca556490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:30.521851 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc831f6-c508-4a1e-a7fc-96d1ca556490 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:30.530874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1323/2675] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:31.990947 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ff22c-88a7-4210-805c-56b0c0d9f38a service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:31.999731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ff22c-88a7-4210-805c-56b0c0d9f38a service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:32.003372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69484e-3658-4718-b003-604e6e313f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:32.004007 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69484e-3658-4718-b003-604e6e313f88 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:32.004285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69484e-3658-4718-b003-604e6e313f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:32.004560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69484e-3658-4718-b003-604e6e313f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:32.014072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69484e-3658-4718-b003-604e6e313f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:32.015478 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69484e-3658-4718-b003-604e6e313f88 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:32.025415 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1353/2676] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:35.933377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc831f6-c508-4a1e-a7fc-96d1ca556490 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:35.934834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4fc831f6-c508-4a1e-a7fc-96d1ca556490 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:35.982155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:36.000247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc831f6-c508-4a1e-a7fc-96d1ca556490 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:36.004745 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:36.005652 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:36.005964 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:36.006495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] PATH_INFO: `/v3/users/b0a9608f59414e8b849774df9cd52a0c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:36.007675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a9608f59414e8b849774df9cd52a0c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:36.009579 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:36.209627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a9608f59414e8b849774df9cd52a0c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:36.210319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a9608f59414e8b849774df9cd52a0c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:36.222557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a9608f59414e8b849774df9cd52a0c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:36.229301 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a9608f59414e8b849774df9cd52a0c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:36.315443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a9608f59414e8b849774df9cd52a0c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:36.321742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1324/2677] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:35 2022] DELETE /identity/v3/users/b0a9608f59414e8b849774df9cd52a0c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:36.326428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e547b6-fc6b-423e-b457-cf3ce7f0dad3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:36.326913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e547b6-fc6b-423e-b457-cf3ce7f0dad3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:36.327029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e547b6-fc6b-423e-b457-cf3ce7f0dad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:36.344333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a2e547b6-fc6b-423e-b457-cf3ce7f0dad3 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:36.345529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a2e547b6-fc6b-423e-b457-cf3ce7f0dad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:36.404495 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1354/2678] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:06:36.414115 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:36.415500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:36.471978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda29973-c522-4042-813f-88951f1ea8ce admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:36.475604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaa3dd-74c4-4f8a-a8a4-1fcde97fbf76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:36.476237 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa3dd-74c4-4f8a-a8a4-1fcde97fbf76 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:36.476518 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa3dd-74c4-4f8a-a8a4-1fcde97fbf76 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:36.476798 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa3dd-74c4-4f8a-a8a4-1fcde97fbf76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:36.485601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaa3dd-74c4-4f8a-a8a4-1fcde97fbf76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:36.487057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaa3dd-74c4-4f8a-a8a4-1fcde97fbf76 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:36.504502 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1325/2679] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:36.826904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e547b6-fc6b-423e-b457-cf3ce7f0dad3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:36.828519 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a2e547b6-fc6b-423e-b457-cf3ce7f0dad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:36.840385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e547b6-fc6b-423e-b457-cf3ce7f0dad3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:36.845054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:36.846018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:36.846305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:36.846587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] PATH_INFO: `/v3/users/01402a48b47a4e25b587387dd18af04f` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:36.847892 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] RBAC: Authorizing `identity:delete_user(user_id=01402a48b47a4e25b587387dd18af04f)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:36.849843 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:36.864426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:36.968508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaa3dd-74c4-4f8a-a8a4-1fcde97fbf76 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:36.970056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7daaa3dd-74c4-4f8a-a8a4-1fcde97fbf76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:36.983655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaa3dd-74c4-4f8a-a8a4-1fcde97fbf76 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:36.988844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:36.989714 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:36.990043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:36.990471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] PATH_INFO: `/v3/projects/6681180051264d52a8fb7b37fabfde87` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:36.991675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6681180051264d52a8fb7b37fabfde87)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:36.993603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:37.152175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01402a48b47a4e25b587387dd18af04f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.152805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6681180051264d52a8fb7b37fabfde87', 'request_id': 'req-dd11ca35-c30f-43c4-a14b-54407da25161'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.153058 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01402a48b47a4e25b587387dd18af04f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.159035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6681180051264d52a8fb7b37fabfde87', 'request_id': 'req-dd11ca35-c30f-43c4-a14b-54407da25161'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.160628 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01402a48b47a4e25b587387dd18af04f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.161707 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:37.172955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01402a48b47a4e25b587387dd18af04f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.223788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6681180051264d52a8fb7b37fabfde87', 'request_id': 'req-dd11ca35-c30f-43c4-a14b-54407da25161'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.241055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] The token cache is being invalidate because project 6681180051264d52a8fb7b37fabfde87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:37.241055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.241055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-dd11ca35-c30f-43c4-a14b-54407da25161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.241055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1326/2680] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:36 2022] DELETE /identity/v3/projects/6681180051264d52a8fb7b37fabfde87 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:06:37.242744 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656e7ae-317b-4228-8d26-167522ffa8f1 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.243035 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656e7ae-317b-4228-8d26-167522ffa8f1 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.243316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656e7ae-317b-4228-8d26-167522ffa8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.261284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9656e7ae-317b-4228-8d26-167522ffa8f1 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:37.262575 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9656e7ae-317b-4228-8d26-167522ffa8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:37.298816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-1c27688e-ebbf-4071-b7b3-099e5b34638c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01402a48b47a4e25b587387dd18af04f'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:37.305255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1355/2681] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:36 2022] DELETE /identity/v3/users/01402a48b47a4e25b587387dd18af04f => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:37.310571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b37a91-4886-4c46-9bbc-330707a0bf42 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.310571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b37a91-4886-4c46-9bbc-330707a0bf42 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.310571 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b37a91-4886-4c46-9bbc-330707a0bf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.319690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1327/2682] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:06:37.327299 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb14d8-5475-47fb-b071-48ffe8d87106 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.327605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb14d8-5475-47fb-b071-48ffe8d87106 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.327734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-87b37a91-4886-4c46-9bbc-330707a0bf42 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:37.328487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb14d8-5475-47fb-b071-48ffe8d87106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.328853 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-87b37a91-4886-4c46-9bbc-330707a0bf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:37.347001 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-d6fb14d8-5475-47fb-b071-48ffe8d87106 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:37.348204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d6fb14d8-5475-47fb-b071-48ffe8d87106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:37.387225 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1356/2683] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:06:37.396863 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b37a91-4886-4c46-9bbc-330707a0bf42 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:37.398388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-87b37a91-4886-4c46-9bbc-330707a0bf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:37.413649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1328/2684] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:37.417549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb14d8-5475-47fb-b071-48ffe8d87106 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:37.427050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb14d8-5475-47fb-b071-48ffe8d87106 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:37.431738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdddfa-9bdd-4cb3-9513-b4d9f7c6dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:37.432394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdddfa-9bdd-4cb3-9513-b4d9f7c6dcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.432681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdddfa-9bdd-4cb3-9513-b4d9f7c6dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.432954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdddfa-9bdd-4cb3-9513-b4d9f7c6dcf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.433699 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdddfa-9bdd-4cb3-9513-b4d9f7c6dcf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:06:37.434031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdddfa-9bdd-4cb3-9513-b4d9f7c6dcf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:37.435635 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cdddfa-9bdd-4cb3-9513-b4d9f7c6dcf9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:37.447743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1329/2685] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:06:37.451914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdddfa-9bdd-4cb3-9513-b4d9f7c6dcf9 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:37.456837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b37a91-4886-4c46-9bbc-330707a0bf42 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:37.460624 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4818b9-da3b-40eb-aadd-9d3788baa5a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:37.461270 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4818b9-da3b-40eb-aadd-9d3788baa5a6 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.461555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4818b9-da3b-40eb-aadd-9d3788baa5a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.461669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdddfa-9bdd-4cb3-9513-b4d9f7c6dcf9 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:37.461907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4818b9-da3b-40eb-aadd-9d3788baa5a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.466104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be6a49-932d-400b-bf62-102efba74d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:37.466727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be6a49-932d-400b-bf62-102efba74d0c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.467008 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be6a49-932d-400b-bf62-102efba74d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.467274 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be6a49-932d-400b-bf62-102efba74d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.467945 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be6a49-932d-400b-bf62-102efba74d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:37.470482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be6a49-932d-400b-bf62-102efba74d0c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:37.471656 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4818b9-da3b-40eb-aadd-9d3788baa5a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:37.473110 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4818b9-da3b-40eb-aadd-9d3788baa5a6 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:37.482845 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1357/2686] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:37.603593 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1330/2687] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:06:37 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:06:37.607737 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4818b9-da3b-40eb-aadd-9d3788baa5a6 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:37.617450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4818b9-da3b-40eb-aadd-9d3788baa5a6 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:37.649836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:37.673998 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4473f8-71ad-4602-a848-b70e889aab9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:37.677839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4473f8-71ad-4602-a848-b70e889aab9d admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.677839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4473f8-71ad-4602-a848-b70e889aab9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.677839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4473f8-71ad-4602-a848-b70e889aab9d admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.677839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4473f8-71ad-4602-a848-b70e889aab9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:37.677839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4473f8-71ad-4602-a848-b70e889aab9d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:37.771049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1358/2688] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:06:37 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:37.772361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be6a49-932d-400b-bf62-102efba74d0c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:37.783733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be6a49-932d-400b-bf62-102efba74d0c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:37.788450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597f6d-7391-458d-9372-f21d66d6322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:37.789116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597f6d-7391-458d-9372-f21d66d6322a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.789409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597f6d-7391-458d-9372-f21d66d6322a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.789687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21597f6d-7391-458d-9372-f21d66d6322a admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.790443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21597f6d-7391-458d-9372-f21d66d6322a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:37.792134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21597f6d-7391-458d-9372-f21d66d6322a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:37.805113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1331/2689] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:37.806479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4473f8-71ad-4602-a848-b70e889aab9d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:37.817558 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4473f8-71ad-4602-a848-b70e889aab9d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:37.822739 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa83b91-840f-4719-af83-13a14bd82aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:37.824234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa83b91-840f-4719-af83-13a14bd82aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.824234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa83b91-840f-4719-af83-13a14bd82aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.824234 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa83b91-840f-4719-af83-13a14bd82aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.824711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa83b91-840f-4719-af83-13a14bd82aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:37.826335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa83b91-840f-4719-af83-13a14bd82aa7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:37.836943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1359/2690] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:37.841224 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597f6d-7391-458d-9372-f21d66d6322a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:37.852302 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21597f6d-7391-458d-9372-f21d66d6322a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:37.857278 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da18cf-387c-48f8-a8d3-32aabab7f5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:37.858202 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da18cf-387c-48f8-a8d3-32aabab7f5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.858511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da18cf-387c-48f8-a8d3-32aabab7f5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.858787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da18cf-387c-48f8-a8d3-32aabab7f5e9 admin admin] PATH_INFO: `/v3/projects/4bf78540156d45c4be84f1e693b620cd/users/840bb17a96ab4ffda471ca4222edd016/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.873276 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da18cf-387c-48f8-a8d3-32aabab7f5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf78540156d45c4be84f1e693b620cd, user_id=840bb17a96ab4ffda471ca4222edd016, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:37.876621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da18cf-387c-48f8-a8d3-32aabab7f5e9 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:37.940194 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa83b91-840f-4719-af83-13a14bd82aa7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:37.942024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1fa83b91-840f-4719-af83-13a14bd82aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:37.956211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa83b91-840f-4719-af83-13a14bd82aa7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:37.960927 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:37.961787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.962108 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.962504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] PATH_INFO: `/v3/projects/ad786d15a1c74e89aa73cb2379cd783a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:37.963701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad786d15a1c74e89aa73cb2379cd783a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:37.965706 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:37.975282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1332/2691] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:37 2022] PUT /identity/v3/projects/4bf78540156d45c4be84f1e693b620cd/users/840bb17a96ab4ffda471ca4222edd016/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:37.987289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d3d84-06c8-4a9a-93c2-cd0c44273694 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:37.987684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d3d84-06c8-4a9a-93c2-cd0c44273694 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:37.987998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d3d84-06c8-4a9a-93c2-cd0c44273694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:38.014289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-849d3d84-06c8-4a9a-93c2-cd0c44273694 None None] MFA Rules not processed for user `840bb17a96ab4ffda471ca4222edd016`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:38.015440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-849d3d84-06c8-4a9a-93c2-cd0c44273694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:38.023928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:38.076914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1333/2692] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:06:38.088253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d3d84-06c8-4a9a-93c2-cd0c44273694 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:38.089656 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-849d3d84-06c8-4a9a-93c2-cd0c44273694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:38.139798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad786d15a1c74e89aa73cb2379cd783a', 'request_id': 'req-78181057-6add-4083-b19b-23cf8e176f59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.146898 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad786d15a1c74e89aa73cb2379cd783a', 'request_id': 'req-78181057-6add-4083-b19b-23cf8e176f59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.149853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d3d84-06c8-4a9a-93c2-cd0c44273694 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:38.196933 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371400c-f4c7-43ee-9e8c-1273ed9374e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:38.197578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371400c-f4c7-43ee-9e8c-1273ed9374e4 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:38.197863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371400c-f4c7-43ee-9e8c-1273ed9374e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:38.198254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371400c-f4c7-43ee-9e8c-1273ed9374e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:38.229789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad786d15a1c74e89aa73cb2379cd783a', 'request_id': 'req-78181057-6add-4083-b19b-23cf8e176f59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.244025 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] The token cache is being invalidate because project ad786d15a1c74e89aa73cb2379cd783a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:38.244319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.244982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.246639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1360/2693] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:37 2022] DELETE /identity/v3/projects/ad786d15a1c74e89aa73cb2379cd783a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:06:38.251139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:38.252491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:38.254710 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371400c-f4c7-43ee-9e8c-1273ed9374e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:38.256439 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371400c-f4c7-43ee-9e8c-1273ed9374e4 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:38.266335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1334/2694] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:06:38.316443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78181057-6add-4083-b19b-23cf8e176f59 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:38.321345 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:38.322300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:38.322535 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:38.322815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] PATH_INFO: `/v3/users/f2bd4d7a2dd04165b6611c13bde6aec1` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:38.324049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2bd4d7a2dd04165b6611c13bde6aec1)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:38.326047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:38.518255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:38.523932 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2bd4d7a2dd04165b6611c13bde6aec1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.524596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2bd4d7a2dd04165b6611c13bde6aec1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.531238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2bd4d7a2dd04165b6611c13bde6aec1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.542562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2bd4d7a2dd04165b6611c13bde6aec1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.623725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2bd4d7a2dd04165b6611c13bde6aec1'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:38.629873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1361/2695] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:38 2022] DELETE /identity/v3/users/f2bd4d7a2dd04165b6611c13bde6aec1 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:39.018219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371400c-f4c7-43ee-9e8c-1273ed9374e4 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:39.043392 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371400c-f4c7-43ee-9e8c-1273ed9374e4 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:39.066977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=100 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:39.107692 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:39.108594 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:39.108878 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:39.109158 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] PATH_INFO: `/v3/projects/36437dba73cf49559ee32d948e480ade` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:39.110397 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36437dba73cf49559ee32d948e480ade)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:39.112308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:39.201313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36437dba73cf49559ee32d948e480ade', 'request_id': 'req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.207471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36437dba73cf49559ee32d948e480ade', 'request_id': 'req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.298377 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36437dba73cf49559ee32d948e480ade', 'request_id': 'req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.309848 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] The token cache is being invalidate because project 36437dba73cf49559ee32d948e480ade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:39.310168 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.310781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.312308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1335/2696] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:39 2022] DELETE /identity/v3/projects/36437dba73cf49559ee32d948e480ade => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:39.316910 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:39.318404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:39.332115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb36d17-0316-4b0f-afe0-c1a69bd09976 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:39.387859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:39.388725 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:39.389011 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:39.389290 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] PATH_INFO: `/v3/users/1aaccfe5e77843f89ab8dba35d1ca3be` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:39.390488 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aaccfe5e77843f89ab8dba35d1ca3be)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:39.393195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:39.569966 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aaccfe5e77843f89ab8dba35d1ca3be'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.570652 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aaccfe5e77843f89ab8dba35d1ca3be'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.577960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aaccfe5e77843f89ab8dba35d1ca3be'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.590864 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aaccfe5e77843f89ab8dba35d1ca3be'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.692348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-a007e232-15aa-4582-8c63-85139831268c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aaccfe5e77843f89ab8dba35d1ca3be'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:39.698885 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1362/2697] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:39 2022] DELETE /identity/v3/users/1aaccfe5e77843f89ab8dba35d1ca3be => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:40.106411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:40.109504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:40.126148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee232e-7a73-4b54-a8b5-d0ce8fea8c30 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:40.189385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:40.190290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.190577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.190856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] PATH_INFO: `/v3/projects/703152913a4c427e8ae4416ff5c3583b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:40.192052 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=703152913a4c427e8ae4416ff5c3583b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:40.194018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:40.326291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703152913a4c427e8ae4416ff5c3583b', 'request_id': 'req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:40.328221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:40.332043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703152913a4c427e8ae4416ff5c3583b', 'request_id': 'req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:40.407340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703152913a4c427e8ae4416ff5c3583b', 'request_id': 'req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:40.419023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] The token cache is being invalidate because project 703152913a4c427e8ae4416ff5c3583b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:40.419311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:40.419940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6d50138f-52aa-4d2a-9d54-a3f4a46808b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:40.421481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1336/2698] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:40 2022] DELETE /identity/v3/projects/703152913a4c427e8ae4416ff5c3583b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:40.427044 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e723-8575-40d1-bed4-cd6cf21b5980 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.427336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e723-8575-40d1-bed4-cd6cf21b5980 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.427616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23e723-8575-40d1-bed4-cd6cf21b5980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:40.445414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-af23e723-8575-40d1-bed4-cd6cf21b5980 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:40.446637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-af23e723-8575-40d1-bed4-cd6cf21b5980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:40.478378 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:40.507160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1363/2699] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:40.516778 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88db43-f069-4cba-b428-5c6d7fcc906c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.517092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88db43-f069-4cba-b428-5c6d7fcc906c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.517374 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88db43-f069-4cba-b428-5c6d7fcc906c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:40.538037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-ec88db43-f069-4cba-b428-5c6d7fcc906c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:40.541634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-ec88db43-f069-4cba-b428-5c6d7fcc906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:40.547242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1337/2700] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:40.551812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23e723-8575-40d1-bed4-cd6cf21b5980 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:40.561407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23e723-8575-40d1-bed4-cd6cf21b5980 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:40.566418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3be66-f5d5-4092-8129-1329a672fa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:40.567072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3be66-f5d5-4092-8129-1329a672fa70 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.567356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3be66-f5d5-4092-8129-1329a672fa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.567639 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3be66-f5d5-4092-8129-1329a672fa70 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:40.568363 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3be66-f5d5-4092-8129-1329a672fa70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:06:40.568669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3be66-f5d5-4092-8129-1329a672fa70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:40.570297 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3be66-f5d5-4092-8129-1329a672fa70 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:40.582623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1364/2701] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:06:40.587131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88db43-f069-4cba-b428-5c6d7fcc906c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:40.597114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88db43-f069-4cba-b428-5c6d7fcc906c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:40.602223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ace61-7921-472a-bcc7-5cbf76dbad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:40.602885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ace61-7921-472a-bcc7-5cbf76dbad46 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.604923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ace61-7921-472a-bcc7-5cbf76dbad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.604923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ace61-7921-472a-bcc7-5cbf76dbad46 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:40.604923 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ace61-7921-472a-bcc7-5cbf76dbad46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:40.606199 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ace61-7921-472a-bcc7-5cbf76dbad46 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:40.725591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1338/2702] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:06:40 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:40.730071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3be66-f5d5-4092-8129-1329a672fa70 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:40.740315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3be66-f5d5-4092-8129-1329a672fa70 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:40.795437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6563f-f69f-440f-a3fe-6dbac1f24dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:40.796102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6563f-f69f-440f-a3fe-6dbac1f24dac admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.796388 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6563f-f69f-440f-a3fe-6dbac1f24dac admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.796670 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6563f-f69f-440f-a3fe-6dbac1f24dac admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:40.797406 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6563f-f69f-440f-a3fe-6dbac1f24dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:40.799384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6563f-f69f-440f-a3fe-6dbac1f24dac admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:40.898969 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1365/2703] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:06:40 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:40.903402 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ace61-7921-472a-bcc7-5cbf76dbad46 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:40.912380 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6563f-f69f-440f-a3fe-6dbac1f24dac admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:40.913991 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ace61-7921-472a-bcc7-5cbf76dbad46 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:40.914136 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-14f6563f-f69f-440f-a3fe-6dbac1f24dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:40.918568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dbed5-80f4-4f27-93dc-b82034a6af5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:40.919254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dbed5-80f4-4f27-93dc-b82034a6af5e admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.919535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dbed5-80f4-4f27-93dc-b82034a6af5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.919863 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dbed5-80f4-4f27-93dc-b82034a6af5e admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:40.920586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dbed5-80f4-4f27-93dc-b82034a6af5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:40.922260 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dbed5-80f4-4f27-93dc-b82034a6af5e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:40.933210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1339/2704] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:40.937243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dbed5-80f4-4f27-93dc-b82034a6af5e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:40.947410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dbed5-80f4-4f27-93dc-b82034a6af5e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:40.952042 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eced3f-46f4-40f4-bf04-d1e75bba0206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:40.952697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eced3f-46f4-40f4-bf04-d1e75bba0206 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.952981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eced3f-46f4-40f4-bf04-d1e75bba0206 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.953958 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eced3f-46f4-40f4-bf04-d1e75bba0206 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:40.954661 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eced3f-46f4-40f4-bf04-d1e75bba0206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:40.956237 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eced3f-46f4-40f4-bf04-d1e75bba0206 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:40.966380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1340/2705] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:06:40.970708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eced3f-46f4-40f4-bf04-d1e75bba0206 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:40.972246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6563f-f69f-440f-a3fe-6dbac1f24dac admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:40.975827 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a9d43-bcbd-4cc9-8ca4-14449e5e3ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:40.976516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a9d43-bcbd-4cc9-8ca4-14449e5e3ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.976797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a9d43-bcbd-4cc9-8ca4-14449e5e3ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.977072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a9d43-bcbd-4cc9-8ca4-14449e5e3ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:40.978896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8b8a9d43-bcbd-4cc9-8ca4-14449e5e3ad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:40.981122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eced3f-46f4-40f4-bf04-d1e75bba0206 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:40.985715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948ce24-9726-4c2c-82af-c5f8270b878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:40.986631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948ce24-9726-4c2c-82af-c5f8270b878b admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:40.995581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948ce24-9726-4c2c-82af-c5f8270b878b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:40.995581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948ce24-9726-4c2c-82af-c5f8270b878b admin admin] PATH_INFO: `/v3/projects/1d061bdf777245329bc5ee39f60f49eb/users/2c6a4b9c60f6447c81fd5f5a3d54ee59/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:41.004447 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948ce24-9726-4c2c-82af-c5f8270b878b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d061bdf777245329bc5ee39f60f49eb, user_id=2c6a4b9c60f6447c81fd5f5a3d54ee59, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:41.007480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0948ce24-9726-4c2c-82af-c5f8270b878b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:41.040667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a9d43-bcbd-4cc9-8ca4-14449e5e3ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:41.042165 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a9d43-bcbd-4cc9-8ca4-14449e5e3ad8 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:41.051882 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1366/2706] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:06:41.121495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1341/2707] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:40 2022] PUT /identity/v3/projects/1d061bdf777245329bc5ee39f60f49eb/users/2c6a4b9c60f6447c81fd5f5a3d54ee59/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:06:41.132094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeec9c-666b-406a-a2fe-2189977d8864 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:41.132870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeec9c-666b-406a-a2fe-2189977d8864 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:41.132870 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeec9c-666b-406a-a2fe-2189977d8864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:41.144210 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:41.159122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-bddeec9c-666b-406a-a2fe-2189977d8864 None None] MFA Rules not processed for user `2c6a4b9c60f6447c81fd5f5a3d54ee59`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:41.160314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bddeec9c-666b-406a-a2fe-2189977d8864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:41.223879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1367/2708] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:06:41.235292 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948ce24-9726-4c2c-82af-c5f8270b878b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:41.236963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0948ce24-9726-4c2c-82af-c5f8270b878b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:41.293396 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948ce24-9726-4c2c-82af-c5f8270b878b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:41.297122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579fab6-d188-4ec6-a7f8-d4f11ce1c486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:41.297794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579fab6-d188-4ec6-a7f8-d4f11ce1c486 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:41.298104 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579fab6-d188-4ec6-a7f8-d4f11ce1c486 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:41.298382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579fab6-d188-4ec6-a7f8-d4f11ce1c486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:41.306694 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579fab6-d188-4ec6-a7f8-d4f11ce1c486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:41.308120 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579fab6-d188-4ec6-a7f8-d4f11ce1c486 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:41.316751 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1342/2709] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:06:46.484816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeec9c-666b-406a-a2fe-2189977d8864 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:46.554699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeec9c-666b-406a-a2fe-2189977d8864 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:46.560817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:46.564391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:46.564391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:46.564391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] PATH_INFO: `/v3/users/2c6a4b9c60f6447c81fd5f5a3d54ee59` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:46.564391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6a4b9c60f6447c81fd5f5a3d54ee59)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:46.565425 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:46.804050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6a4b9c60f6447c81fd5f5a3d54ee59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:46.804704 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6a4b9c60f6447c81fd5f5a3d54ee59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:46.814392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6a4b9c60f6447c81fd5f5a3d54ee59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:46.826823 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6a4b9c60f6447c81fd5f5a3d54ee59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:46.898384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6a4b9c60f6447c81fd5f5a3d54ee59'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:46.904874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1368/2710] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:46 2022] DELETE /identity/v3/users/2c6a4b9c60f6447c81fd5f5a3d54ee59 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:46.909735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf880db6-8a9e-45e5-9699-431abb0e47a0 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:46.910239 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf880db6-8a9e-45e5-9699-431abb0e47a0 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:46.910335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf880db6-8a9e-45e5-9699-431abb0e47a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:46.928334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-bf880db6-8a9e-45e5-9699-431abb0e47a0 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:46.929584 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bf880db6-8a9e-45e5-9699-431abb0e47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:46.988904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1343/2711] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:46.998662 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:47.002060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:47.020818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:47.057767 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc348c19-2551-4bbb-9755-e38228fc216f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:47.061970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792e5d4-df56-4f97-a9e9-e3973388f937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:47.062615 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792e5d4-df56-4f97-a9e9-e3973388f937 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:47.062894 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792e5d4-df56-4f97-a9e9-e3973388f937 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:47.063562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792e5d4-df56-4f97-a9e9-e3973388f937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:47.073420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792e5d4-df56-4f97-a9e9-e3973388f937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:47.074875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792e5d4-df56-4f97-a9e9-e3973388f937 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:47.085056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1369/2712] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:47.671505 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf880db6-8a9e-45e5-9699-431abb0e47a0 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:47.672976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-bf880db6-8a9e-45e5-9699-431abb0e47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:47.696356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf880db6-8a9e-45e5-9699-431abb0e47a0 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:47.712957 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:47.713850 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:47.714169 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:47.714451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] PATH_INFO: `/v3/projects/1d061bdf777245329bc5ee39f60f49eb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:47.716310 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d061bdf777245329bc5ee39f60f49eb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:47.718330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:47.721410 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:47.869284 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d061bdf777245329bc5ee39f60f49eb', 'request_id': 'req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:47.876897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d061bdf777245329bc5ee39f60f49eb', 'request_id': 'req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:47.989726 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d061bdf777245329bc5ee39f60f49eb', 'request_id': 'req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:48.003433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] The token cache is being invalidate because project 1d061bdf777245329bc5ee39f60f49eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:48.003725 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:48.004384 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1a1f616f-c68d-4cb0-9c17-4d57a9b987a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:48.006277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1344/2713] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:47 2022] DELETE /identity/v3/projects/1d061bdf777245329bc5ee39f60f49eb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:48.011229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24569c84-86dd-4a7c-a91a-24ac74ef1315 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:48.011692 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24569c84-86dd-4a7c-a91a-24ac74ef1315 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:48.011809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24569c84-86dd-4a7c-a91a-24ac74ef1315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:48.031348 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-24569c84-86dd-4a7c-a91a-24ac74ef1315 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:48.032562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-24569c84-86dd-4a7c-a91a-24ac74ef1315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:48.097512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1370/2714] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:48.105227 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873812b-0c86-4c9a-a0b1-a1a3f1f226e6 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:48.105554 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873812b-0c86-4c9a-a0b1-a1a3f1f226e6 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:48.105996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873812b-0c86-4c9a-a0b1-a1a3f1f226e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:48.125003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-0873812b-0c86-4c9a-a0b1-a1a3f1f226e6 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:48.126382 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0873812b-0c86-4c9a-a0b1-a1a3f1f226e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:48.134636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1345/2715] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:48.138890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24569c84-86dd-4a7c-a91a-24ac74ef1315 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:48.149000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24569c84-86dd-4a7c-a91a-24ac74ef1315 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:48.153653 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1d9fe-9921-4182-a5c1-45d2fe0a2e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:48.154334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1d9fe-9921-4182-a5c1-45d2fe0a2e4b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:48.154623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1d9fe-9921-4182-a5c1-45d2fe0a2e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:48.154954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1d9fe-9921-4182-a5c1-45d2fe0a2e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:48.155681 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1d9fe-9921-4182-a5c1-45d2fe0a2e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:06:48.155993 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1d9fe-9921-4182-a5c1-45d2fe0a2e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:48.158195 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1d9fe-9921-4182-a5c1-45d2fe0a2e4b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:48.169850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1371/2716] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:06:48.174247 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873812b-0c86-4c9a-a0b1-a1a3f1f226e6 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:48.183465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873812b-0c86-4c9a-a0b1-a1a3f1f226e6 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:48.188203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c524c-66aa-4215-a443-204bcd94c58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:48.188886 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c524c-66aa-4215-a443-204bcd94c58a admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:48.189174 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c524c-66aa-4215-a443-204bcd94c58a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:48.189458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c524c-66aa-4215-a443-204bcd94c58a admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:48.190205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c524c-66aa-4215-a443-204bcd94c58a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:48.192113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c524c-66aa-4215-a443-204bcd94c58a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:48.297490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1346/2717] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:06:48 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:48.298716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1d9fe-9921-4182-a5c1-45d2fe0a2e4b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:48.308612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1d9fe-9921-4182-a5c1-45d2fe0a2e4b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:48.366531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cf49b-e30f-45fa-b2ea-221343c59291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:48.367118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cf49b-e30f-45fa-b2ea-221343c59291 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:48.367402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cf49b-e30f-45fa-b2ea-221343c59291 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:48.367677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cf49b-e30f-45fa-b2ea-221343c59291 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:48.368356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cf49b-e30f-45fa-b2ea-221343c59291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:48.370280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cf49b-e30f-45fa-b2ea-221343c59291 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:48.479146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1372/2718] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:06:48 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:06:48.483064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c524c-66aa-4215-a443-204bcd94c58a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:48.501843 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c524c-66aa-4215-a443-204bcd94c58a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:48.528455 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bb70c-581f-4dd4-b8a6-7ccb493cc94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:48.534075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb70c-581f-4dd4-b8a6-7ccb493cc94a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:48.534075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb70c-581f-4dd4-b8a6-7ccb493cc94a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:48.534075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb70c-581f-4dd4-b8a6-7ccb493cc94a admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:48.534075 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bb70c-581f-4dd4-b8a6-7ccb493cc94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:48.536215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bb70c-581f-4dd4-b8a6-7ccb493cc94a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:48.561743 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1347/2719] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:06:48.563821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cf49b-e30f-45fa-b2ea-221343c59291 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:48.573988 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cf49b-e30f-45fa-b2ea-221343c59291 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:48.579403 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816de46-d954-44b3-95e1-7a06222548c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:48.582273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816de46-d954-44b3-95e1-7a06222548c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:48.582273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816de46-d954-44b3-95e1-7a06222548c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:48.582273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816de46-d954-44b3-95e1-7a06222548c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:48.582273 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816de46-d954-44b3-95e1-7a06222548c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:48.582986 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816de46-d954-44b3-95e1-7a06222548c0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:48.593485 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1373/2720] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:06:48.598127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bb70c-581f-4dd4-b8a6-7ccb493cc94a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:48.609079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bb70c-581f-4dd4-b8a6-7ccb493cc94a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:48.614721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fd850-1bd7-4bc7-82bb-6b9929283a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:48.615623 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fd850-1bd7-4bc7-82bb-6b9929283a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:48.615910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fd850-1bd7-4bc7-82bb-6b9929283a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:48.616195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fd850-1bd7-4bc7-82bb-6b9929283a36 admin admin] PATH_INFO: `/v3/projects/81b63bf161074034b8ea5cbceccf3feb/users/74491bb24cad41d5a5b05815a401ddf3/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:48.633927 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fd850-1bd7-4bc7-82bb-6b9929283a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b63bf161074034b8ea5cbceccf3feb, user_id=74491bb24cad41d5a5b05815a401ddf3, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:48.636154 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fd850-1bd7-4bc7-82bb-6b9929283a36 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:48.811811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1348/2721] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:06:48 2022] PUT /identity/v3/projects/81b63bf161074034b8ea5cbceccf3feb/users/74491bb24cad41d5a5b05815a401ddf3/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:48.813330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa483b72-c4e6-4730-829d-47bd3a74c1df None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:48.813655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa483b72-c4e6-4730-829d-47bd3a74c1df None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:48.813977 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa483b72-c4e6-4730-829d-47bd3a74c1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:48.847978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-fa483b72-c4e6-4730-829d-47bd3a74c1df None None] MFA Rules not processed for user `74491bb24cad41d5a5b05815a401ddf3`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:48.849838 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-fa483b72-c4e6-4730-829d-47bd3a74c1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:48.862917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:48.923218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1374/2722] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:06:48.936140 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fd850-1bd7-4bc7-82bb-6b9929283a36 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:48.937979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-935fd850-1bd7-4bc7-82bb-6b9929283a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:49.003147 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fd850-1bd7-4bc7-82bb-6b9929283a36 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:49.008445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6fcc1-554b-43d4-81ae-306f5b195656 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd59a351971e94aa38a22e8113207e5b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:49.009330 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6fcc1-554b-43d4-81ae-306f5b195656 service glance] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:49.009734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6fcc1-554b-43d4-81ae-306f5b195656 service glance] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:49.010243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6fcc1-554b-43d4-81ae-306f5b195656 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:49.022334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6fcc1-554b-43d4-81ae-306f5b195656 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:49.024206 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6fcc1-554b-43d4-81ae-306f5b195656 service glance] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:49.038355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1349/2723] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:49.670169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c827f8-6ec0-495f-9807-d034545cf5c3 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:49.670671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c827f8-6ec0-495f-9807-d034545cf5c3 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:49.670781 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c827f8-6ec0-495f-9807-d034545cf5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:49.687896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-e0c827f8-6ec0-495f-9807-d034545cf5c3 None None] MFA Rules not processed for user `646fecf4405c41a0bc062c6b54fd311c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:49.689084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e0c827f8-6ec0-495f-9807-d034545cf5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:49.743896 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1375/2724] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:51.687051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6fcc1-554b-43d4-81ae-306f5b195656 service glance] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:51.688451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-eba6fcc1-554b-43d4-81ae-306f5b195656 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:51.726380 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:51.749073 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6fcc1-554b-43d4-81ae-306f5b195656 service glance] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:51.753370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:51.754046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:51.754340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:51.754619 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:51.756642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:51.815205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:51.816573 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:51.825363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1350/2725] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:52.395677 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c827f8-6ec0-495f-9807-d034545cf5c3 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:52.397097 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e0c827f8-6ec0-495f-9807-d034545cf5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:52.462959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:52.464506 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:52.470949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c827f8-6ec0-495f-9807-d034545cf5c3 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:52.476102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:52.476758 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:52.477070 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:52.477356 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:52.479200 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:52.480701 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1247c-3d50-4132-8276-43f7599e7fac service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:52.486642 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6a0fb-4d17-4b6c-ac01-645dc565001a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:52.487508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6a0fb-4d17-4b6c-ac01-645dc565001a service placement] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:52.487859 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6a0fb-4d17-4b6c-ac01-645dc565001a service placement] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:52.488277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6a0fb-4d17-4b6c-ac01-645dc565001a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:52.493614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:52.495367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:52.507131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6a0fb-4d17-4b6c-ac01-645dc565001a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:52.508836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6a0fb-4d17-4b6c-ac01-645dc565001a service placement] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:52.508967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1376/2726] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:52.526059 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1351/2727] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:52.913751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:52.915380 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:52.968372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:52.984063 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eae984-e494-4529-940f-864a9a1cfce7 service placement] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:52.989417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:52.990336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:52.990622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:52.990981 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] PATH_INFO: `/v3/users/840bb17a96ab4ffda471ca4222edd016` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:52.992215 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=840bb17a96ab4ffda471ca4222edd016)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:52.994229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:53.193809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840bb17a96ab4ffda471ca4222edd016'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:53.194412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840bb17a96ab4ffda471ca4222edd016'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:53.201579 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840bb17a96ab4ffda471ca4222edd016'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:53.218159 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840bb17a96ab4ffda471ca4222edd016'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:53.298241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840bb17a96ab4ffda471ca4222edd016'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:53.304717 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1377/2728] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:06:52 2022] DELETE /identity/v3/users/840bb17a96ab4ffda471ca4222edd016 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:06:53.309423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d5604-55b7-4659-b5b6-0a25d6c10771 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:53.309950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d5604-55b7-4659-b5b6-0a25d6c10771 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:53.310088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d5604-55b7-4659-b5b6-0a25d6c10771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:53.329062 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-f21d5604-55b7-4659-b5b6-0a25d6c10771 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:06:53.331874 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f21d5604-55b7-4659-b5b6-0a25d6c10771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:53.396700 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1352/2729] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:06:53.406436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:53.408538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:53.471890 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a4c68-3a94-417f-b73d-837d2a926c2b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:53.475715 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ca1df-d730-481e-80b6-ad87152f0ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:53.476313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ca1df-d730-481e-80b6-ad87152f0ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:53.476600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ca1df-d730-481e-80b6-ad87152f0ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:53.476877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ca1df-d730-481e-80b6-ad87152f0ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:53.487494 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ca1df-d730-481e-80b6-ad87152f0ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:53.489111 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ca1df-d730-481e-80b6-ad87152f0ba4 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:53.499096 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1378/2730] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:06:54.100170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d5604-55b7-4659-b5b6-0a25d6c10771 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:54.102097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-f21d5604-55b7-4659-b5b6-0a25d6c10771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:54.118780 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d5604-55b7-4659-b5b6-0a25d6c10771 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:54.125308 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:54.126906 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:54.127574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:54.134713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] PATH_INFO: `/v3/projects/4bf78540156d45c4be84f1e693b620cd` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:54.134713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf78540156d45c4be84f1e693b620cd)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:54.134713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:54.241079 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ca1df-d730-481e-80b6-ad87152f0ba4 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:06:54.242603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4d2ca1df-d730-481e-80b6-ad87152f0ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:54.256809 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ca1df-d730-481e-80b6-ad87152f0ba4 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:06:54.261604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:06:54.262332 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:06:54.262669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:06:54.262949 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:06:54.265064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:06:54.280054 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:06:54.281682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:06:54.284004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf78540156d45c4be84f1e693b620cd', 'request_id': 'req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:54.290996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf78540156d45c4be84f1e693b620cd', 'request_id': 'req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:54.294351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1379/2731] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:06:54.446585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf78540156d45c4be84f1e693b620cd', 'request_id': 'req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:54.448823 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:06:54.458541 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] The token cache is being invalidate because project 4bf78540156d45c4be84f1e693b620cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:06:54.458939 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:54.459650 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:06:54.461277 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1353/2732] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:06:54 2022] DELETE /identity/v3/projects/4bf78540156d45c4be84f1e693b620cd => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:02.337907 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:02.339313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:02.352019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfe698-bae6-437b-8031-63dc142a694b service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:02.402636 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:02.408163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:02.409022 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:02.409338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:02.409612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] PATH_INFO: `/v3/users/5d332241d6f84959880bfa99a24805dc` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:02.410808 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d332241d6f84959880bfa99a24805dc)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:02.412695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:02.678146 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d332241d6f84959880bfa99a24805dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:02.678720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d332241d6f84959880bfa99a24805dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:02.685521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d332241d6f84959880bfa99a24805dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:02.697393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d332241d6f84959880bfa99a24805dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:02.772023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d332241d6f84959880bfa99a24805dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:02.778603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1380/2733] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:07:02 2022] DELETE /identity/v3/users/5d332241d6f84959880bfa99a24805dc => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:03.071037 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:03.073131 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:03.089483 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2aab1-d727-4965-8e53-ad9b0dc369c6 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:03.151122 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:03.151988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:03.152273 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:03.152554 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] PATH_INFO: `/v3/projects/3c44b2575d1945e0a100fe081e5af582` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:03.153840 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c44b2575d1945e0a100fe081e5af582)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:03.155889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:03.280489 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c44b2575d1945e0a100fe081e5af582', 'request_id': 'req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.288784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c44b2575d1945e0a100fe081e5af582', 'request_id': 'req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.475221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c44b2575d1945e0a100fe081e5af582', 'request_id': 'req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.487289 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] The token cache is being invalidate because project 3c44b2575d1945e0a100fe081e5af582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:07:03.487578 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.488220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.490023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1354/2734] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:07:03 2022] DELETE /identity/v3/projects/3c44b2575d1945e0a100fe081e5af582 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:03.493837 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:03.495367 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:03.509512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423007db-64f0-4131-ae05-15910b5a55d1 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:03.565115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:03.566019 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:03.566395 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:03.566564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] PATH_INFO: `/v3/users/8c496c477f8a47998f0466595519acbf` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:03.567763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c496c477f8a47998f0466595519acbf)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:03.569665 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:03.774606 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c496c477f8a47998f0466595519acbf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.775262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c496c477f8a47998f0466595519acbf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.782661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c496c477f8a47998f0466595519acbf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.794451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c496c477f8a47998f0466595519acbf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.798512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:03.874851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c496c477f8a47998f0466595519acbf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:03.885167 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1381/2735] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:07:03 2022] DELETE /identity/v3/users/8c496c477f8a47998f0466595519acbf => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:04.135293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:04.135293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:04.158901 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e7fc5-d3e1-418d-a91b-b4450c4be9c8 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:04.227623 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:04.228620 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:04.228903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:04.229633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] PATH_INFO: `/v3/projects/c01f79bef61c457a93cd4d4ce289cf54` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:04.230877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c01f79bef61c457a93cd4d4ce289cf54)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:04.232802 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:04.340740 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:04.354984 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c01f79bef61c457a93cd4d4ce289cf54', 'request_id': 'req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:04.360275 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c01f79bef61c457a93cd4d4ce289cf54', 'request_id': 'req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:04.440545 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c01f79bef61c457a93cd4d4ce289cf54', 'request_id': 'req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:04.454098 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] The token cache is being invalidate because project c01f79bef61c457a93cd4d4ce289cf54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:07:04.454399 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:04.455047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-60ffd75c-fd64-4ffe-9306-4e8f83ed0739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:04.456592 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1355/2736] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:07:04 2022] DELETE /identity/v3/projects/c01f79bef61c457a93cd4d4ce289cf54 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:14.212339 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:14.487182 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:18.672300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:18.673822 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:18.686449 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5856ad-c465-47c4-822e-e53cdf80042a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:18.742731 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:18.743604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:18.743888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:18.744163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] PATH_INFO: `/v3/users/74491bb24cad41d5a5b05815a401ddf3` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:18.745338 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74491bb24cad41d5a5b05815a401ddf3)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:18.747295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:18.933509 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74491bb24cad41d5a5b05815a401ddf3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:18.934113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74491bb24cad41d5a5b05815a401ddf3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:18.941743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74491bb24cad41d5a5b05815a401ddf3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:18.953563 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74491bb24cad41d5a5b05815a401ddf3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:19.017305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74491bb24cad41d5a5b05815a401ddf3'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:19.024030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1382/2737] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:07:18 2022] DELETE /identity/v3/users/74491bb24cad41d5a5b05815a401ddf3 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:19.028568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f53da-0193-4b47-956f-9fc15bca5300 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:19.034637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f53da-0193-4b47-956f-9fc15bca5300 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:19.034637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f53da-0193-4b47-956f-9fc15bca5300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:19.046791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3c7f53da-0193-4b47-956f-9fc15bca5300 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:19.047981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3c7f53da-0193-4b47-956f-9fc15bca5300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:19.106532 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1356/2738] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:19.116123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:19.117548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:19.174134 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29c03f-0080-4728-90db-782e97eb9de5 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:19.177730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e0ebb-df30-496d-9bfd-35e7aa16134a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:19.178429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0ebb-df30-496d-9bfd-35e7aa16134a service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:19.178713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0ebb-df30-496d-9bfd-35e7aa16134a service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:19.178984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0ebb-df30-496d-9bfd-35e7aa16134a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:19.187982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e0ebb-df30-496d-9bfd-35e7aa16134a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:19.189521 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e0ebb-df30-496d-9bfd-35e7aa16134a service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:19.199352 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1383/2739] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:07:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:07:19.607959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f53da-0193-4b47-956f-9fc15bca5300 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:19.609480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3c7f53da-0193-4b47-956f-9fc15bca5300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:19.621477 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f53da-0193-4b47-956f-9fc15bca5300 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:19.626315 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:19.627189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:19.627508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:19.627831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] PATH_INFO: `/v3/projects/81b63bf161074034b8ea5cbceccf3feb` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:19.629108 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b63bf161074034b8ea5cbceccf3feb)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:19.631361 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:19.772623 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b63bf161074034b8ea5cbceccf3feb', 'request_id': 'req-243efda6-1f24-4796-b1bb-521eab4bfca1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:19.783002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b63bf161074034b8ea5cbceccf3feb', 'request_id': 'req-243efda6-1f24-4796-b1bb-521eab4bfca1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:19.920577 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b63bf161074034b8ea5cbceccf3feb', 'request_id': 'req-243efda6-1f24-4796-b1bb-521eab4bfca1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:19.932654 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] The token cache is being invalidate because project 81b63bf161074034b8ea5cbceccf3feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:07:19.932875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:19.933488 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-243efda6-1f24-4796-b1bb-521eab4bfca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:19.935066 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1357/2740] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:07:19 2022] DELETE /identity/v3/projects/81b63bf161074034b8ea5cbceccf3feb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:19.939914 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9377-cddf-4616-9a75-70f35df0bf09 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:19.940414 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9377-cddf-4616-9a75-70f35df0bf09 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:19.940544 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a9377-cddf-4616-9a75-70f35df0bf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:19.958371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-040a9377-cddf-4616-9a75-70f35df0bf09 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:19.959668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-040a9377-cddf-4616-9a75-70f35df0bf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:19.977537 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:20.018815 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1384/2741] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:20.026762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d342c-46e6-4f31-a089-32b590d8c97c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:20.027074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d342c-46e6-4f31-a089-32b590d8c97c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:20.027376 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d342c-46e6-4f31-a089-32b590d8c97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:20.046225 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-4d5d342c-46e6-4f31-a089-32b590d8c97c None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:20.047436 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4d5d342c-46e6-4f31-a089-32b590d8c97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:20.055538 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1358/2742] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:20.059744 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9377-cddf-4616-9a75-70f35df0bf09 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:20.069582 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a9377-cddf-4616-9a75-70f35df0bf09 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:20.074493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f64a5-b676-4ac4-97d9-9cc5b19c3ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:20.075148 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f64a5-b676-4ac4-97d9-9cc5b19c3ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:20.075438 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f64a5-b676-4ac4-97d9-9cc5b19c3ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:20.075719 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f64a5-b676-4ac4-97d9-9cc5b19c3ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:20.076474 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f64a5-b676-4ac4-97d9-9cc5b19c3ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:07:20.076783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f64a5-b676-4ac4-97d9-9cc5b19c3ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:20.078407 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f64a5-b676-4ac4-97d9-9cc5b19c3ce8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:20.090457 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1385/2743] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:07:20.094988 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d342c-46e6-4f31-a089-32b590d8c97c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:20.104834 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d342c-46e6-4f31-a089-32b590d8c97c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:20.109540 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cdc77-c859-431c-91cf-a9c050934ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:20.110223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cdc77-c859-431c-91cf-a9c050934ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:20.110513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cdc77-c859-431c-91cf-a9c050934ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:20.110795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cdc77-c859-431c-91cf-a9c050934ee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:20.111490 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cdc77-c859-431c-91cf-a9c050934ee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:20.113393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cdc77-c859-431c-91cf-a9c050934ee5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:20.271251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1359/2744] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:07:20 2022] POST /identity/v3/projects => generated 391 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:07:20.275313 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f64a5-b676-4ac4-97d9-9cc5b19c3ce8 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:20.285605 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f64a5-b676-4ac4-97d9-9cc5b19c3ce8 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:20.338392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d45dbb-2bb2-4419-b2f5-9bfd0898ab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:20.339047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d45dbb-2bb2-4419-b2f5-9bfd0898ab97 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:20.339328 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d45dbb-2bb2-4419-b2f5-9bfd0898ab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:20.339604 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d45dbb-2bb2-4419-b2f5-9bfd0898ab97 admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:20.340318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d45dbb-2bb2-4419-b2f5-9bfd0898ab97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:20.342281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d45dbb-2bb2-4419-b2f5-9bfd0898ab97 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:20.480578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1386/2745] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:07:20 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:07:20.484753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cdc77-c859-431c-91cf-a9c050934ee5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:20.495357 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cdc77-c859-431c-91cf-a9c050934ee5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:20.499979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02464cd0-ba3a-49b5-915c-a0f13f689a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:20.500634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02464cd0-ba3a-49b5-915c-a0f13f689a43 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:20.500918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02464cd0-ba3a-49b5-915c-a0f13f689a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:20.501201 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02464cd0-ba3a-49b5-915c-a0f13f689a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:20.502053 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02464cd0-ba3a-49b5-915c-a0f13f689a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:20.504209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02464cd0-ba3a-49b5-915c-a0f13f689a43 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:20.514897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1360/2746] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:07:20.518735 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d45dbb-2bb2-4419-b2f5-9bfd0898ab97 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:20.529632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d45dbb-2bb2-4419-b2f5-9bfd0898ab97 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:20.535491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13b832-6115-4dfa-a8ac-9d998db4d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:20.536233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13b832-6115-4dfa-a8ac-9d998db4d00d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:20.536502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13b832-6115-4dfa-a8ac-9d998db4d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:20.536782 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13b832-6115-4dfa-a8ac-9d998db4d00d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:20.537568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13b832-6115-4dfa-a8ac-9d998db4d00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:20.539241 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13b832-6115-4dfa-a8ac-9d998db4d00d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:20.549361 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1387/2747] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:07:20.553533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02464cd0-ba3a-49b5-915c-a0f13f689a43 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:20.563487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02464cd0-ba3a-49b5-915c-a0f13f689a43 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:20.568517 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca493f-9678-4547-b193-f50afd524288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:20.569423 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca493f-9678-4547-b193-f50afd524288 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:20.569713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca493f-9678-4547-b193-f50afd524288 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:20.570038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca493f-9678-4547-b193-f50afd524288 admin admin] PATH_INFO: `/v3/projects/2520970511224763853b5497421870d5/users/bd30380bcd7d49bc9a1ad8de6227e501/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:20.584738 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca493f-9678-4547-b193-f50afd524288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2520970511224763853b5497421870d5, user_id=bd30380bcd7d49bc9a1ad8de6227e501, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:20.587433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca493f-9678-4547-b193-f50afd524288 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:20.698508 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1361/2748] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:07:20 2022] PUT /identity/v3/projects/2520970511224763853b5497421870d5/users/bd30380bcd7d49bc9a1ad8de6227e501/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:20.710746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22897cc5-ad1f-4d63-afcc-944ff439b412 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:20.711039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22897cc5-ad1f-4d63-afcc-944ff439b412 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:20.711318 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22897cc5-ad1f-4d63-afcc-944ff439b412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:20.740497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-22897cc5-ad1f-4d63-afcc-944ff439b412 None None] MFA Rules not processed for user `bd30380bcd7d49bc9a1ad8de6227e501`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:20.743684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-22897cc5-ad1f-4d63-afcc-944ff439b412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:20.806771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1388/2749] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:20.818313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca493f-9678-4547-b193-f50afd524288 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:20.820985 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-27ca493f-9678-4547-b193-f50afd524288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:20.842535 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:20.878471 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca493f-9678-4547-b193-f50afd524288 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:20.882812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5709eb-2432-420b-9102-3516f07b119e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:20.883474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5709eb-2432-420b-9102-3516f07b119e service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:20.883760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5709eb-2432-420b-9102-3516f07b119e service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:20.884041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5709eb-2432-420b-9102-3516f07b119e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:20.892978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5709eb-2432-420b-9102-3516f07b119e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:20.895034 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5709eb-2432-420b-9102-3516f07b119e service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:20.904257 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1362/2750] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:07:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:07:30.803746 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:30.983628 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=17 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:35.224980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91411d97-54b4-432a-b71d-685406566a4b None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:35.225524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91411d97-54b4-432a-b71d-685406566a4b None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:35.225524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91411d97-54b4-432a-b71d-685406566a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:35.243941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-91411d97-54b4-432a-b71d-685406566a4b None None] MFA Rules not processed for user `ce77221a2c8d4c4395bed092c1967b28`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:35.245124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-91411d97-54b4-432a-b71d-685406566a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:35.298085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1389/2751] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:42.186385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5709eb-2432-420b-9102-3516f07b119e service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:42.247430 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5709eb-2432-420b-9102-3516f07b119e service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:42.251989 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:42.252857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:42.253143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:42.253428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] PATH_INFO: `/v3/users/bd30380bcd7d49bc9a1ad8de6227e501` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:42.254649 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd30380bcd7d49bc9a1ad8de6227e501)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:42.256839 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:42.442242 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd30380bcd7d49bc9a1ad8de6227e501'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:42.442899 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd30380bcd7d49bc9a1ad8de6227e501'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:42.454771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd30380bcd7d49bc9a1ad8de6227e501'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:42.461451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd30380bcd7d49bc9a1ad8de6227e501'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:42.560078 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd30380bcd7d49bc9a1ad8de6227e501'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:42.566458 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1363/2752] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:07:42 2022] DELETE /identity/v3/users/bd30380bcd7d49bc9a1ad8de6227e501 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:42.571060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f118ff-b0ed-46d0-a6d6-2ac1861a4598 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:42.571564 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f118ff-b0ed-46d0-a6d6-2ac1861a4598 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:42.571667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f118ff-b0ed-46d0-a6d6-2ac1861a4598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:42.589214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-13f118ff-b0ed-46d0-a6d6-2ac1861a4598 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:42.590446 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-13f118ff-b0ed-46d0-a6d6-2ac1861a4598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:42.649023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1390/2753] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:42.658609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:42.660056 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:42.717580 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a021a6-080a-4d10-bb99-e7604b47a8cb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:42.720856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4f7e9-5587-41e9-be72-7e5caefd574a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:42.721504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4f7e9-5587-41e9-be72-7e5caefd574a service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:42.721789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4f7e9-5587-41e9-be72-7e5caefd574a service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:42.722095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4f7e9-5587-41e9-be72-7e5caefd574a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:42.731348 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4f7e9-5587-41e9-be72-7e5caefd574a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:42.732782 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4f7e9-5587-41e9-be72-7e5caefd574a service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:42.742873 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1364/2754] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:07:43.169944 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f118ff-b0ed-46d0-a6d6-2ac1861a4598 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:43.177678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-13f118ff-b0ed-46d0-a6d6-2ac1861a4598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:43.183687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f118ff-b0ed-46d0-a6d6-2ac1861a4598 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:43.188299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:43.189156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:43.189439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:43.189718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] PATH_INFO: `/v3/projects/2520970511224763853b5497421870d5` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:43.191310 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2520970511224763853b5497421870d5)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:43.192839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:43.293463 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2520970511224763853b5497421870d5', 'request_id': 'req-62d99f7e-5582-4c51-a6ff-03fc623fcc15'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:43.299136 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2520970511224763853b5497421870d5', 'request_id': 'req-62d99f7e-5582-4c51-a6ff-03fc623fcc15'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:43.354252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2520970511224763853b5497421870d5', 'request_id': 'req-62d99f7e-5582-4c51-a6ff-03fc623fcc15'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:43.365715 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] The token cache is being invalidate because project 2520970511224763853b5497421870d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:07:43.366039 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:43.366669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-62d99f7e-5582-4c51-a6ff-03fc623fcc15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:07:43.368305 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1391/2755] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:07:43 2022] DELETE /identity/v3/projects/2520970511224763853b5497421870d5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:43.373250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a5cc-a588-4c45-abee-4f29c23b2a53 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:43.373729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a5cc-a588-4c45-abee-4f29c23b2a53 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:43.373853 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a5cc-a588-4c45-abee-4f29c23b2a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:43.391266 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7d64a5cc-a588-4c45-abee-4f29c23b2a53 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:43.392461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7d64a5cc-a588-4c45-abee-4f29c23b2a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:43.409677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:43.452554 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1365/2756] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:07:43.460851 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf693ee-1fbc-4aa0-9ed7-b8b877b3be75 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:43.461143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf693ee-1fbc-4aa0-9ed7-b8b877b3be75 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:43.461428 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf693ee-1fbc-4aa0-9ed7-b8b877b3be75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:43.479819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-9bf693ee-1fbc-4aa0-9ed7-b8b877b3be75 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:43.481049 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-9bf693ee-1fbc-4aa0-9ed7-b8b877b3be75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:43.488849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1392/2757] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:43.496091 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a5cc-a588-4c45-abee-4f29c23b2a53 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:43.506016 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a5cc-a588-4c45-abee-4f29c23b2a53 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:43.510877 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740edb59-b263-4abf-8616-d6ae86cb6585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:43.511542 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740edb59-b263-4abf-8616-d6ae86cb6585 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:43.511827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740edb59-b263-4abf-8616-d6ae86cb6585 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:43.512105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740edb59-b263-4abf-8616-d6ae86cb6585 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:43.512829 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740edb59-b263-4abf-8616-d6ae86cb6585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:07:43.513385 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740edb59-b263-4abf-8616-d6ae86cb6585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:43.514904 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740edb59-b263-4abf-8616-d6ae86cb6585 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:43.527254 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1366/2758] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:07:43.534914 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf693ee-1fbc-4aa0-9ed7-b8b877b3be75 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:43.544799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf693ee-1fbc-4aa0-9ed7-b8b877b3be75 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:43.549559 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55635688-d0d5-4e8b-8729-35e63eb51534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:43.550244 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55635688-d0d5-4e8b-8729-35e63eb51534 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:43.550532 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55635688-d0d5-4e8b-8729-35e63eb51534 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:43.550812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55635688-d0d5-4e8b-8729-35e63eb51534 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:43.551516 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55635688-d0d5-4e8b-8729-35e63eb51534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:43.553484 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55635688-d0d5-4e8b-8729-35e63eb51534 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:43.641678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:43.648920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1393/2759] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:07:43 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:07:43.653023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740edb59-b263-4abf-8616-d6ae86cb6585 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:43.664339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740edb59-b263-4abf-8616-d6ae86cb6585 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:43.724509 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbeea62-dbd1-4064-8d85-90fa68752ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:43.725191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeea62-dbd1-4064-8d85-90fa68752ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:43.725478 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeea62-dbd1-4064-8d85-90fa68752ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:43.725758 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeea62-dbd1-4064-8d85-90fa68752ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:43.726498 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbeea62-dbd1-4064-8d85-90fa68752ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:43.728417 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbeea62-dbd1-4064-8d85-90fa68752ed5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:43.850335 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1367/2760] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:07:43 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:07:43.851730 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55635688-d0d5-4e8b-8729-35e63eb51534 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:43.869309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55635688-d0d5-4e8b-8729-35e63eb51534 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:43.873810 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d558f07-22b8-4dc7-af0c-81054ab7ee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:43.874499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d558f07-22b8-4dc7-af0c-81054ab7ee7a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:43.874785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d558f07-22b8-4dc7-af0c-81054ab7ee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:43.875066 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d558f07-22b8-4dc7-af0c-81054ab7ee7a admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:43.876020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d558f07-22b8-4dc7-af0c-81054ab7ee7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:43.877734 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d558f07-22b8-4dc7-af0c-81054ab7ee7a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:43.888663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1394/2761] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:07:43.892640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbeea62-dbd1-4064-8d85-90fa68752ed5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:43.902644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbeea62-dbd1-4064-8d85-90fa68752ed5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:43.907360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ed522-f93d-4cb3-a977-99322c8f2484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:43.908032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ed522-f93d-4cb3-a977-99322c8f2484 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:43.908318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ed522-f93d-4cb3-a977-99322c8f2484 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:43.908601 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ed522-f93d-4cb3-a977-99322c8f2484 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:43.909355 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ed522-f93d-4cb3-a977-99322c8f2484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:43.911523 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ed522-f93d-4cb3-a977-99322c8f2484 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:43.921711 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1368/2762] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:07:43.925770 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d558f07-22b8-4dc7-af0c-81054ab7ee7a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:43.936181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d558f07-22b8-4dc7-af0c-81054ab7ee7a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:43.940631 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddfda8-68dd-4413-a1a7-5685b4103c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:43.944467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddfda8-68dd-4413-a1a7-5685b4103c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:43.944467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddfda8-68dd-4413-a1a7-5685b4103c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:43.944467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddfda8-68dd-4413-a1a7-5685b4103c8e admin admin] PATH_INFO: `/v3/projects/bc1aca15157e45bf95284345dcabf101/users/0b94fa871aee4c35851e6e939838d2f6/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:43.956238 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddfda8-68dd-4413-a1a7-5685b4103c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1aca15157e45bf95284345dcabf101, user_id=0b94fa871aee4c35851e6e939838d2f6, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:43.958959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddfda8-68dd-4413-a1a7-5685b4103c8e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:44.017391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1395/2763] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:07:43 2022] PUT /identity/v3/projects/bc1aca15157e45bf95284345dcabf101/users/0b94fa871aee4c35851e6e939838d2f6/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:07:44.022946 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960be72-1b1e-426e-991e-f5d22f5e3d4a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:44.023236 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960be72-1b1e-426e-991e-f5d22f5e3d4a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:44.023510 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960be72-1b1e-426e-991e-f5d22f5e3d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:44.041634 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3960be72-1b1e-426e-991e-f5d22f5e3d4a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:44.042854 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3960be72-1b1e-426e-991e-f5d22f5e3d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:44.104040 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1369/2764] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:44.114118 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddfda8-68dd-4413-a1a7-5685b4103c8e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:44.115866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-d0ddfda8-68dd-4413-a1a7-5685b4103c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:44.172920 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddfda8-68dd-4413-a1a7-5685b4103c8e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:44.176726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d65bd-d4c8-49c7-91e6-18a55616e436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:44.179477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d65bd-d4c8-49c7-91e6-18a55616e436 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:44.179477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d65bd-d4c8-49c7-91e6-18a55616e436 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:44.179477 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d65bd-d4c8-49c7-91e6-18a55616e436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:44.187515 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d65bd-d4c8-49c7-91e6-18a55616e436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:44.188970 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d65bd-d4c8-49c7-91e6-18a55616e436 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:44.198439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1396/2765] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:07:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:07:49.632185 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376274b8-68e6-4e4d-8d84-bf445afb74fd None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:49.633081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376274b8-68e6-4e4d-8d84-bf445afb74fd None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:49.633568 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376274b8-68e6-4e4d-8d84-bf445afb74fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:49.667796 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-376274b8-68e6-4e4d-8d84-bf445afb74fd None None] MFA Rules not processed for user `0b94fa871aee4c35851e6e939838d2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:49.669043 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-376274b8-68e6-4e4d-8d84-bf445afb74fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:49.689599 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:49.773171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1370/2766] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:49.777821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d65bd-d4c8-49c7-91e6-18a55616e436 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:49.789383 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d65bd-d4c8-49c7-91e6-18a55616e436 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:49.796680 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f841fc-1095-4f16-9289-7dc3fce5600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:49.797464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f841fc-1095-4f16-9289-7dc3fce5600c admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:49.798030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f841fc-1095-4f16-9289-7dc3fce5600c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:49.798539 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f841fc-1095-4f16-9289-7dc3fce5600c admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:49.799322 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f841fc-1095-4f16-9289-7dc3fce5600c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:49.801408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f841fc-1095-4f16-9289-7dc3fce5600c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:49.895751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1397/2767] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:07:49 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:07:49.897609 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376274b8-68e6-4e4d-8d84-bf445afb74fd None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:49.919145 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376274b8-68e6-4e4d-8d84-bf445afb74fd None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:50.000667 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49c32a-4f4a-4354-a9ff-871961f8fc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:50.001445 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49c32a-4f4a-4354-a9ff-871961f8fc9c admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:50.001832 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49c32a-4f4a-4354-a9ff-871961f8fc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:50.002240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49c32a-4f4a-4354-a9ff-871961f8fc9c admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:50.003005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49c32a-4f4a-4354-a9ff-871961f8fc9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:50.005464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49c32a-4f4a-4354-a9ff-871961f8fc9c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:50.115797 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1371/2768] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:07:49 2022] POST /identity/v3/users => generated 299 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:07:50.117471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f841fc-1095-4f16-9289-7dc3fce5600c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:50.149548 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f841fc-1095-4f16-9289-7dc3fce5600c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:50.162805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7fa9-adf4-4a2a-ac1a-f9e3f5be125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:50.163517 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7fa9-adf4-4a2a-ac1a-f9e3f5be125f admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:50.168797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7fa9-adf4-4a2a-ac1a-f9e3f5be125f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:50.168797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb7fa9-adf4-4a2a-ac1a-f9e3f5be125f admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:50.168797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb7fa9-adf4-4a2a-ac1a-f9e3f5be125f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:50.168797 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb7fa9-adf4-4a2a-ac1a-f9e3f5be125f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:50.186040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1398/2769] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:07:50.189981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49c32a-4f4a-4354-a9ff-871961f8fc9c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:50.201856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49c32a-4f4a-4354-a9ff-871961f8fc9c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:50.209226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113cd87-5347-4781-b1ab-16b37748230f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:50.210391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113cd87-5347-4781-b1ab-16b37748230f admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:50.211875 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113cd87-5347-4781-b1ab-16b37748230f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:50.213457 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113cd87-5347-4781-b1ab-16b37748230f admin admin] PATH_INFO: `/v3/projects/10c669a8ac59456fbf4e8bac2135a2a1/users/c151f74afd1d47838ef11241f3d93f06/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:50.233818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113cd87-5347-4781-b1ab-16b37748230f admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c669a8ac59456fbf4e8bac2135a2a1, user_id=c151f74afd1d47838ef11241f3d93f06, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:50.236733 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113cd87-5347-4781-b1ab-16b37748230f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:50.315536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1372/2770] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:07:50 2022] PUT /identity/v3/projects/10c669a8ac59456fbf4e8bac2135a2a1/users/c151f74afd1d47838ef11241f3d93f06/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:07:50.320362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7fa9-adf4-4a2a-ac1a-f9e3f5be125f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:50.331821 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb7fa9-adf4-4a2a-ac1a-f9e3f5be125f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:50.391083 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e837b-6030-45a5-a5f4-607877d47f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:50.391762 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e837b-6030-45a5-a5f4-607877d47f8a admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:50.400086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e837b-6030-45a5-a5f4-607877d47f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:50.400086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e837b-6030-45a5-a5f4-607877d47f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:50.400086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e837b-6030-45a5-a5f4-607877d47f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:50.400086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e837b-6030-45a5-a5f4-607877d47f8a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:50.409741 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1399/2771] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:07:50.410974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113cd87-5347-4781-b1ab-16b37748230f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:50.421860 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113cd87-5347-4781-b1ab-16b37748230f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:50.427944 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ba096-3009-4553-97a4-671554624329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:50.428870 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba096-3009-4553-97a4-671554624329 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:50.434600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba096-3009-4553-97a4-671554624329 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:50.434600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba096-3009-4553-97a4-671554624329 admin admin] PATH_INFO: `/v3/projects/10c669a8ac59456fbf4e8bac2135a2a1/users/c151f74afd1d47838ef11241f3d93f06/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:50.434600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ba096-3009-4553-97a4-671554624329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c669a8ac59456fbf4e8bac2135a2a1, user_id=c151f74afd1d47838ef11241f3d93f06, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:50.434600 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ba096-3009-4553-97a4-671554624329 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:50.446618 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1373/2772] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:07:50 2022] PUT /identity/v3/projects/10c669a8ac59456fbf4e8bac2135a2a1/users/c151f74afd1d47838ef11241f3d93f06/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:07:55.994610 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bb05b-bb74-447e-9f7d-17157c5107de None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:55.995117 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bb05b-bb74-447e-9f7d-17157c5107de None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:55.995218 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bb05b-bb74-447e-9f7d-17157c5107de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:56.023877 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-807bb05b-bb74-447e-9f7d-17157c5107de None None] MFA Rules not processed for user `c151f74afd1d47838ef11241f3d93f06`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:56.025086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-807bb05b-bb74-447e-9f7d-17157c5107de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:56.044300 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:56.094667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1400/2773] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:07:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:56.106391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ba096-3009-4553-97a4-671554624329 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:56.108156 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fb5ba096-3009-4553-97a4-671554624329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:56.167729 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ba096-3009-4553-97a4-671554624329 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:56.171294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d222d7-1b10-4214-b648-38bb6bcb20da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd59a351971e94aa38a22e8113207e5b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:56.171937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d222d7-1b10-4214-b648-38bb6bcb20da service glance] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:56.172219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d222d7-1b10-4214-b648-38bb6bcb20da service glance] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:56.172495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d222d7-1b10-4214-b648-38bb6bcb20da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:56.198055 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:07:56.230482 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d222d7-1b10-4214-b648-38bb6bcb20da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:56.231903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d222d7-1b10-4214-b648-38bb6bcb20da service glance] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:56.244908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1374/2774] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:07:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:07:56.516875 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea7db2-cbef-4c86-9d17-d4dc56d3adf9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:56.518283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea7db2-cbef-4c86-9d17-d4dc56d3adf9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:56.518283 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea7db2-cbef-4c86-9d17-d4dc56d3adf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:56.536062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-56ea7db2-cbef-4c86-9d17-d4dc56d3adf9 None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:07:56.537264 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-56ea7db2-cbef-4c86-9d17-d4dc56d3adf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:56.597181 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1401/2775] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 10:07:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:07:56.735720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d222d7-1b10-4214-b648-38bb6bcb20da service glance] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:07:56.737132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-67d222d7-1b10-4214-b648-38bb6bcb20da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:07:56.809812 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d222d7-1b10-4214-b648-38bb6bcb20da service glance] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:07:56.815045 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291b885-b064-4fb3-9e70-ff426c7f6655 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:07:56.815715 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291b885-b064-4fb3-9e70-ff426c7f6655 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:07:56.816003 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291b885-b064-4fb3-9e70-ff426c7f6655 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:07:56.816282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291b885-b064-4fb3-9e70-ff426c7f6655 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:07:56.825409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291b885-b064-4fb3-9e70-ff426c7f6655 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:07:56.827036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291b885-b064-4fb3-9e70-ff426c7f6655 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:07:56.835557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1375/2776] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:07:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:08:06.589747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=45 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:08:06.832028 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=48 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:08:07.769192 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368681c6-2c74-4256-a1ed-c8c34f623c2c None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:07.769682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368681c6-2c74-4256-a1ed-c8c34f623c2c None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:07.769814 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368681c6-2c74-4256-a1ed-c8c34f623c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:07.788086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-368681c6-2c74-4256-a1ed-c8c34f623c2c None None] MFA Rules not processed for user `0b94fa871aee4c35851e6e939838d2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:08:07.789282 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-368681c6-2c74-4256-a1ed-c8c34f623c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:07.796295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1402/2777] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:08:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:08:11.261215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e47258-439a-4541-b221-30777ae316ed None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:11.261791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e47258-439a-4541-b221-30777ae316ed None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:11.261791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e47258-439a-4541-b221-30777ae316ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:11.279816 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-35e47258-439a-4541-b221-30777ae316ed None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:08:11.281103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-35e47258-439a-4541-b221-30777ae316ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:11.288734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1376/2778] 10.222.0.42 () {58 vars in 1009 bytes} [Fri Jun 3 10:08:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:08:11.301206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368681c6-2c74-4256-a1ed-c8c34f623c2c None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:11.314005 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368681c6-2c74-4256-a1ed-c8c34f623c2c None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:11.319641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248a119-2b5c-480c-a956-381d4e7075bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:11.320527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248a119-2b5c-480c-a956-381d4e7075bf service swift] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:11.321006 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248a119-2b5c-480c-a956-381d4e7075bf service swift] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:11.321392 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248a119-2b5c-480c-a956-381d4e7075bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:11.329609 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248a119-2b5c-480c-a956-381d4e7075bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:11.331206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248a119-2b5c-480c-a956-381d4e7075bf service swift] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:11.339311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1403/2779] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:08:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:08:13.020195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e47258-439a-4541-b221-30777ae316ed None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:13.083464 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e47258-439a-4541-b221-30777ae316ed None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:13.087128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116ceff-c2aa-4329-9965-382668ee3079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:13.087785 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116ceff-c2aa-4329-9965-382668ee3079 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:13.088070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116ceff-c2aa-4329-9965-382668ee3079 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:13.088351 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116ceff-c2aa-4329-9965-382668ee3079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:13.090220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0116ceff-c2aa-4329-9965-382668ee3079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:13.168731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116ceff-c2aa-4329-9965-382668ee3079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:13.170209 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116ceff-c2aa-4329-9965-382668ee3079 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:13.181537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1377/2780] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:08:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:08:14.384874 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248a119-2b5c-480c-a956-381d4e7075bf service swift] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:14.395144 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248a119-2b5c-480c-a956-381d4e7075bf service swift] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:14.399482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:14.400138 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:14.400426 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:14.400708 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:14.402577 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:14.417115 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:14.418751 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:14.431829 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1404/2781] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:08:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:08:23.176062 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=57 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:08:24.425817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=15 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:08:43.772219 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116ceff-c2aa-4329-9965-382668ee3079 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:43.780941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116ceff-c2aa-4329-9965-382668ee3079 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:43.785908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f8bac-6954-42c3-9509-ba549a3cf33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:43.785908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f8bac-6954-42c3-9509-ba549a3cf33e service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:43.785908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f8bac-6954-42c3-9509-ba549a3cf33e service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:43.785908 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f8bac-6954-42c3-9509-ba549a3cf33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:43.795581 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f8bac-6954-42c3-9509-ba549a3cf33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:43.797006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f8bac-6954-42c3-9509-ba549a3cf33e service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:43.807162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1378/2782] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:08:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:08:46.207193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:46.210113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:46.225780 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51a8cf-b34b-4ac5-8fdb-a758ea0d6132 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:46.231888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051153-a217-40a2-957f-20b5fe35dd89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:46.232473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051153-a217-40a2-957f-20b5fe35dd89 service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:46.232963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051153-a217-40a2-957f-20b5fe35dd89 service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:46.233413 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051153-a217-40a2-957f-20b5fe35dd89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:46.235491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c4051153-a217-40a2-957f-20b5fe35dd89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:46.248634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4051153-a217-40a2-957f-20b5fe35dd89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:46.250510 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4051153-a217-40a2-957f-20b5fe35dd89 service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:46.262333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1405/2783] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:08:46.369703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f8bac-6954-42c3-9509-ba549a3cf33e service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:46.379687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f8bac-6954-42c3-9509-ba549a3cf33e service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:46.383552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:46.384344 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:46.384769 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:46.385240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:46.388144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:46.400133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:46.401940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:46.411109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1379/2784] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:08:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:08:48.103062 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051153-a217-40a2-957f-20b5fe35dd89 service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:48.115122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051153-a217-40a2-957f-20b5fe35dd89 service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:48.121072 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfe8ef12db914a9693ef470f1706bb5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:48.121701 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:48.122024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:48.122298 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:48.124400 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:48.190796 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:48.192594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:48.202959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1406/2785] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:08:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:08:54.507195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:54.508691 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:54.520210 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd708-f93d-49a7-bb40-10279afa227b service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:54.524696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:54.525337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:54.525622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:54.525950 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:54.527827 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:54.589537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:54.591005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:54.600533 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1380/2786] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:08:58.098667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:58.101293 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:58.173043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce09934-b90d-4f16-830b-4c76298c8acf service nova] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:58.178066 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dfa15-15f1-409e-abaa-7a723730f769 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:58.178707 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dfa15-15f1-409e-abaa-7a723730f769 service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:58.178984 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dfa15-15f1-409e-abaa-7a723730f769 service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:58.179260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dfa15-15f1-409e-abaa-7a723730f769 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:58.181077 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-b76dfa15-15f1-409e-abaa-7a723730f769 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:58.192421 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dfa15-15f1-409e-abaa-7a723730f769 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:58.199288 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dfa15-15f1-409e-abaa-7a723730f769 service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:58.205836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1407/2787] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:08:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:08:58.276704 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:58.278297 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:58.291215 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80ffd3-1e8f-4f86-8c70-1e923fa73c27 service neutron] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:58.295836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:58.296719 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:58.297032 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:58.297354 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] PATH_INFO: `/v3/users/ce77221a2c8d4c4395bed092c1967b28` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:58.298550 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce77221a2c8d4c4395bed092c1967b28)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:58.300462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:58.498858 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce77221a2c8d4c4395bed092c1967b28'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:58.499522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce77221a2c8d4c4395bed092c1967b28'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:58.512359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce77221a2c8d4c4395bed092c1967b28'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:58.519865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce77221a2c8d4c4395bed092c1967b28'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:58.588004 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-92eff822-8860-4d23-9af6-deee08b57105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce77221a2c8d4c4395bed092c1967b28'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:58.594807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1381/2788] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:08:58 2022] DELETE /identity/v3/users/ce77221a2c8d4c4395bed092c1967b28 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:08:58.852330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dfa15-15f1-409e-abaa-7a723730f769 service placement] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:58.863089 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dfa15-15f1-409e-abaa-7a723730f769 service placement] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:58.938487 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:58.939648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:58.939929 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:58.940212 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] PATH_INFO: `/v3/projects/3e51010dd5d84daf8aa43859e9e7cb6d` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:58.941429 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e51010dd5d84daf8aa43859e9e7cb6d)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:58.943945 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:58.961588 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=100 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:08:59.069757 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e51010dd5d84daf8aa43859e9e7cb6d', 'request_id': 'req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:59.077263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e51010dd5d84daf8aa43859e9e7cb6d', 'request_id': 'req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:59.199451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e51010dd5d84daf8aa43859e9e7cb6d', 'request_id': 'req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:59.213170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] The token cache is being invalidate because project 3e51010dd5d84daf8aa43859e9e7cb6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:08:59.213462 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:59.214143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3e2277a1-5253-4e13-81bc-b2ba3a7c6429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:08:59.215795 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1408/2789] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:08:58 2022] DELETE /identity/v3/projects/3e51010dd5d84daf8aa43859e9e7cb6d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:08:59.220890 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcccf2-120b-43e6-9a35-1f426ecfa94a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:59.221212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcccf2-120b-43e6-9a35-1f426ecfa94a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:59.221493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcccf2-120b-43e6-9a35-1f426ecfa94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:59.239955 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-9fbcccf2-120b-43e6-9a35-1f426ecfa94a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:08:59.241775 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-9fbcccf2-120b-43e6-9a35-1f426ecfa94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:59.278304 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:08:59.307011 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1382/2790] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:08:59.314753 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3133-884d-41f1-965d-61303aa057b8 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:59.315037 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3133-884d-41f1-965d-61303aa057b8 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:59.315324 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3133-884d-41f1-965d-61303aa057b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:59.335315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6a9a3133-884d-41f1-965d-61303aa057b8 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:08:59.336567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6a9a3133-884d-41f1-965d-61303aa057b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:08:59.344694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1409/2791] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:08:59.349208 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcccf2-120b-43e6-9a35-1f426ecfa94a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:59.360288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcccf2-120b-43e6-9a35-1f426ecfa94a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:59.365249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa5b72-34f6-4e1c-ba7e-1b09a364919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:59.365948 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa5b72-34f6-4e1c-ba7e-1b09a364919a admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:59.366238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa5b72-34f6-4e1c-ba7e-1b09a364919a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:59.366519 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa5b72-34f6-4e1c-ba7e-1b09a364919a admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:59.367256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa5b72-34f6-4e1c-ba7e-1b09a364919a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:08:59.367563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa5b72-34f6-4e1c-ba7e-1b09a364919a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:59.369212 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa5b72-34f6-4e1c-ba7e-1b09a364919a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:59.385688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1383/2792] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:08:59.386980 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a3133-884d-41f1-965d-61303aa057b8 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:59.397295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a3133-884d-41f1-965d-61303aa057b8 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:59.402048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92f0de-92c1-496b-b6f9-ce2736716535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:59.402718 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f0de-92c1-496b-b6f9-ce2736716535 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:59.403000 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f0de-92c1-496b-b6f9-ce2736716535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:59.403279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f0de-92c1-496b-b6f9-ce2736716535 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:59.403982 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92f0de-92c1-496b-b6f9-ce2736716535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:59.405936 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92f0de-92c1-496b-b6f9-ce2736716535 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:59.492056 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1410/2793] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:08:59 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:08:59.496841 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa5b72-34f6-4e1c-ba7e-1b09a364919a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:59.511676 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa5b72-34f6-4e1c-ba7e-1b09a364919a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:59.601326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97286f18-73d8-437e-8247-7ca2398cdb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:59.601326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97286f18-73d8-437e-8247-7ca2398cdb7e admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:59.601326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97286f18-73d8-437e-8247-7ca2398cdb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:59.601326 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97286f18-73d8-437e-8247-7ca2398cdb7e admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:59.601973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97286f18-73d8-437e-8247-7ca2398cdb7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:59.601973 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97286f18-73d8-437e-8247-7ca2398cdb7e admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:59.697929 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1384/2794] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:08:59 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:08:59.701418 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92f0de-92c1-496b-b6f9-ce2736716535 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:59.715334 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92f0de-92c1-496b-b6f9-ce2736716535 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:59.718320 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574f42-bc5a-44f0-868a-cd97d541e8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:59.719113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574f42-bc5a-44f0-868a-cd97d541e8cc admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:59.719866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574f42-bc5a-44f0-868a-cd97d541e8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:59.719866 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574f42-bc5a-44f0-868a-cd97d541e8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:59.720553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad574f42-bc5a-44f0-868a-cd97d541e8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:59.722381 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad574f42-bc5a-44f0-868a-cd97d541e8cc admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:59.734939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1411/2795] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:08:59.739285 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97286f18-73d8-437e-8247-7ca2398cdb7e admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:59.749772 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97286f18-73d8-437e-8247-7ca2398cdb7e admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:59.754998 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c843f0-cc60-44d7-9b78-6b33e80d27da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:59.756294 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c843f0-cc60-44d7-9b78-6b33e80d27da admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:59.756697 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c843f0-cc60-44d7-9b78-6b33e80d27da admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:59.757070 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c843f0-cc60-44d7-9b78-6b33e80d27da admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:59.757856 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c843f0-cc60-44d7-9b78-6b33e80d27da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:59.759560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c843f0-cc60-44d7-9b78-6b33e80d27da admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:59.770677 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1385/2796] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:08:59.775202 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574f42-bc5a-44f0-868a-cd97d541e8cc admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:08:59.785859 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574f42-bc5a-44f0-868a-cd97d541e8cc admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:08:59.793333 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb4c4d-0d1f-43fe-bedc-0443bdfe97b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:08:59.794901 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb4c4d-0d1f-43fe-bedc-0443bdfe97b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:08:59.795197 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb4c4d-0d1f-43fe-bedc-0443bdfe97b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:08:59.795479 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb4c4d-0d1f-43fe-bedc-0443bdfe97b0 admin admin] PATH_INFO: `/v3/projects/6ca531e0b7b74d6c817bbdfa4bbd796a/users/bc21bcda39084a8e908e360c50f307dc/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:08:59.822124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb4c4d-0d1f-43fe-bedc-0443bdfe97b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca531e0b7b74d6c817bbdfa4bbd796a, user_id=bc21bcda39084a8e908e360c50f307dc, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:08:59.825016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb4c4d-0d1f-43fe-bedc-0443bdfe97b0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:08:59.995904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1412/2797] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:08:59 2022] PUT /identity/v3/projects/6ca531e0b7b74d6c817bbdfa4bbd796a/users/bc21bcda39084a8e908e360c50f307dc/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:00.007214 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be2269-663a-4a10-be83-b22b2607ea34 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:00.007530 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be2269-663a-4a10-be83-b22b2607ea34 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:00.007828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be2269-663a-4a10-be83-b22b2607ea34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:00.047448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-58be2269-663a-4a10-be83-b22b2607ea34 None None] MFA Rules not processed for user `bc21bcda39084a8e908e360c50f307dc`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:00.049456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-58be2269-663a-4a10-be83-b22b2607ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:00.131820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1386/2798] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:00.136232 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb4c4d-0d1f-43fe-bedc-0443bdfe97b0 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:00.215251 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb4c4d-0d1f-43fe-bedc-0443bdfe97b0 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:00.219975 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de330609-8692-4e83-b9d6-7c555ddf1579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:00.220676 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de330609-8692-4e83-b9d6-7c555ddf1579 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:00.222312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de330609-8692-4e83-b9d6-7c555ddf1579 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:00.222599 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de330609-8692-4e83-b9d6-7c555ddf1579 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:00.223294 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de330609-8692-4e83-b9d6-7c555ddf1579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:00.225745 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de330609-8692-4e83-b9d6-7c555ddf1579 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:00.356226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1413/2799] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:09:00 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:00.360585 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be2269-663a-4a10-be83-b22b2607ea34 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:00.371869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be2269-663a-4a10-be83-b22b2607ea34 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:00.404480 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:00.429364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd2c6-fc25-44d5-9c9b-4b1096caa970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:00.430077 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd2c6-fc25-44d5-9c9b-4b1096caa970 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:00.430362 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd2c6-fc25-44d5-9c9b-4b1096caa970 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:00.430636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd2c6-fc25-44d5-9c9b-4b1096caa970 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:00.431334 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdd2c6-fc25-44d5-9c9b-4b1096caa970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:00.433307 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdd2c6-fc25-44d5-9c9b-4b1096caa970 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:00.583181 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1387/2800] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:09:00 2022] POST /identity/v3/users => generated 299 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:09:00.587672 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de330609-8692-4e83-b9d6-7c555ddf1579 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:00.599064 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de330609-8692-4e83-b9d6-7c555ddf1579 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:00.603763 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb014990-6f98-4186-9499-dcda3d725820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:00.604543 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb014990-6f98-4186-9499-dcda3d725820 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:00.604855 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb014990-6f98-4186-9499-dcda3d725820 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:00.605131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb014990-6f98-4186-9499-dcda3d725820 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:00.605854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb014990-6f98-4186-9499-dcda3d725820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:00.607500 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb014990-6f98-4186-9499-dcda3d725820 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:00.619279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1414/2801] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:00.623524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd2c6-fc25-44d5-9c9b-4b1096caa970 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:00.634456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd2c6-fc25-44d5-9c9b-4b1096caa970 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:00.639240 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d1572-8a95-4bd6-b2b2-5a07309d2beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:00.640150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d1572-8a95-4bd6-b2b2-5a07309d2beb admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:00.640434 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d1572-8a95-4bd6-b2b2-5a07309d2beb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:00.640713 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d1572-8a95-4bd6-b2b2-5a07309d2beb admin admin] PATH_INFO: `/v3/projects/60fc2b51ab8f46ae996ea37ef470ed73/users/7c8ad4911fda414280b0391a48597b8a/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:00.657258 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d1572-8a95-4bd6-b2b2-5a07309d2beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=60fc2b51ab8f46ae996ea37ef470ed73, user_id=7c8ad4911fda414280b0391a48597b8a, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:00.659965 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d1572-8a95-4bd6-b2b2-5a07309d2beb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:00.765486 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1388/2802] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:00 2022] PUT /identity/v3/projects/60fc2b51ab8f46ae996ea37ef470ed73/users/7c8ad4911fda414280b0391a48597b8a/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:00.769531 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb014990-6f98-4186-9499-dcda3d725820 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:00.779663 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb014990-6f98-4186-9499-dcda3d725820 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:00.792023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:00.836384 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f6d17-2a01-474e-9477-bccc0c291515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:00.837042 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f6d17-2a01-474e-9477-bccc0c291515 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:00.837330 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f6d17-2a01-474e-9477-bccc0c291515 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:00.837632 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f6d17-2a01-474e-9477-bccc0c291515 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:00.838412 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f6d17-2a01-474e-9477-bccc0c291515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:00.840043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f6d17-2a01-474e-9477-bccc0c291515 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:00.850285 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1415/2803] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:00.854370 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d1572-8a95-4bd6-b2b2-5a07309d2beb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:00.864437 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d1572-8a95-4bd6-b2b2-5a07309d2beb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:00.868975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6037c3-7475-4df5-b768-7eec4337f22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:00.869843 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6037c3-7475-4df5-b768-7eec4337f22b admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:00.870155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6037c3-7475-4df5-b768-7eec4337f22b admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:00.870428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6037c3-7475-4df5-b768-7eec4337f22b admin admin] PATH_INFO: `/v3/projects/60fc2b51ab8f46ae996ea37ef470ed73/users/7c8ad4911fda414280b0391a48597b8a/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:00.872364 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6037c3-7475-4df5-b768-7eec4337f22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60fc2b51ab8f46ae996ea37ef470ed73, user_id=7c8ad4911fda414280b0391a48597b8a, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:00.875025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6037c3-7475-4df5-b768-7eec4337f22b admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:00.885409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1389/2804] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:00 2022] PUT /identity/v3/projects/60fc2b51ab8f46ae996ea37ef470ed73/users/7c8ad4911fda414280b0391a48597b8a/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:00.895344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccc0b7-a19d-4224-8f49-87a3b23800ce None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:00.895633 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccc0b7-a19d-4224-8f49-87a3b23800ce None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:00.895912 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccc0b7-a19d-4224-8f49-87a3b23800ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:00.922265 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-41ccc0b7-a19d-4224-8f49-87a3b23800ce None None] MFA Rules not processed for user `7c8ad4911fda414280b0391a48597b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:00.923541 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-41ccc0b7-a19d-4224-8f49-87a3b23800ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:00.989280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1416/2805] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:01.001226 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6037c3-7475-4df5-b768-7eec4337f22b admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:01.003086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7a6037c3-7475-4df5-b768-7eec4337f22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:01.060910 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6037c3-7475-4df5-b768-7eec4337f22b admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:01.064640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9228ab-953f-4317-8573-014f8831255e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:01.065279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9228ab-953f-4317-8573-014f8831255e service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:01.065586 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9228ab-953f-4317-8573-014f8831255e service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:01.065869 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9228ab-953f-4317-8573-014f8831255e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:01.075057 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9228ab-953f-4317-8573-014f8831255e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:01.076485 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9228ab-953f-4317-8573-014f8831255e service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:01.086588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1390/2806] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:01.114454 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccc0b7-a19d-4224-8f49-87a3b23800ce None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:01.178567 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccc0b7-a19d-4224-8f49-87a3b23800ce None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:01.183282 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:01.184149 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:01.184437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:01.184716 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] PATH_INFO: `/v3/users/bc21bcda39084a8e908e360c50f307dc` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:01.185957 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc21bcda39084a8e908e360c50f307dc)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:01.187879 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:01.424909 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:01.431193 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc21bcda39084a8e908e360c50f307dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:01.431839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc21bcda39084a8e908e360c50f307dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:01.438555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc21bcda39084a8e908e360c50f307dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:01.450634 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc21bcda39084a8e908e360c50f307dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:01.558812 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc21bcda39084a8e908e360c50f307dc'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:01.565336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1417/2807] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:01 2022] DELETE /identity/v3/users/bc21bcda39084a8e908e360c50f307dc => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:01.573088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22f1b6-e49f-42a2-a202-347f846acd0a None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:01.573572 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22f1b6-e49f-42a2-a202-347f846acd0a None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:01.573960 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22f1b6-e49f-42a2-a202-347f846acd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:01.596116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-3b22f1b6-e49f-42a2-a202-347f846acd0a None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:01.597500 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3b22f1b6-e49f-42a2-a202-347f846acd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:01.668666 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1391/2808] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:01.675657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:01.677105 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:01.741708 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69b6d-5789-4f85-a7b2-18736515cdb0 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:01.745295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d395b8-889e-41af-b4ac-16ef55a4ef28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:01.746050 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d395b8-889e-41af-b4ac-16ef55a4ef28 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:01.746396 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d395b8-889e-41af-b4ac-16ef55a4ef28 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:01.746678 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d395b8-889e-41af-b4ac-16ef55a4ef28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:01.757660 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d395b8-889e-41af-b4ac-16ef55a4ef28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:01.759311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d395b8-889e-41af-b4ac-16ef55a4ef28 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:01.770131 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1418/2809] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:02.280889 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22f1b6-e49f-42a2-a202-347f846acd0a None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:02.285771 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-3b22f1b6-e49f-42a2-a202-347f846acd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:02.296113 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22f1b6-e49f-42a2-a202-347f846acd0a None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:02.300688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:02.301549 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:02.301831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:02.302127 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] PATH_INFO: `/v3/projects/6ca531e0b7b74d6c817bbdfa4bbd796a` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:02.304029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca531e0b7b74d6c817bbdfa4bbd796a)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:02.305979 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:02.435216 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:02.455928 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca531e0b7b74d6c817bbdfa4bbd796a', 'request_id': 'req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:02.462339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca531e0b7b74d6c817bbdfa4bbd796a', 'request_id': 'req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:02.564220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca531e0b7b74d6c817bbdfa4bbd796a', 'request_id': 'req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:02.577638 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] The token cache is being invalidate because project 6ca531e0b7b74d6c817bbdfa4bbd796a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:02.578409 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:02.578936 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:02.580880 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1392/2810] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:02 2022] DELETE /identity/v3/projects/6ca531e0b7b74d6c817bbdfa4bbd796a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:02.584816 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d395b8-889e-41af-b4ac-16ef55a4ef28 service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:02.596143 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d395b8-889e-41af-b4ac-16ef55a4ef28 service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:02.654198 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:02.655071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:02.655350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:02.655623 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] PATH_INFO: `/v3/users/7c8ad4911fda414280b0391a48597b8a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:02.656801 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c8ad4911fda414280b0391a48597b8a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:02.658729 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:02.922314 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c8ad4911fda414280b0391a48597b8a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:02.922881 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c8ad4911fda414280b0391a48597b8a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:02.930835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c8ad4911fda414280b0391a48597b8a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:02.937351 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:02.950163 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c8ad4911fda414280b0391a48597b8a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:03.022771 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-b1808a60-2ea2-42de-be58-4224bc700bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c8ad4911fda414280b0391a48597b8a'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:03.029189 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1419/2811] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:02 2022] DELETE /identity/v3/users/7c8ad4911fda414280b0391a48597b8a => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:09:03.468130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:03.468130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:03.480116 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a6483-e994-4cf8-a4e7-a028d1ddbd7a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:03.558119 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:03.559097 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:03.559288 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:03.559552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] PATH_INFO: `/v3/projects/60fc2b51ab8f46ae996ea37ef470ed73` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:03.561132 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] RBAC: Authorizing `identity:delete_project(project_id=60fc2b51ab8f46ae996ea37ef470ed73)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:03.563076 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:03.711019 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60fc2b51ab8f46ae996ea37ef470ed73', 'request_id': 'req-d33545c2-deff-45db-a9cb-d9387ef4612f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:03.717111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60fc2b51ab8f46ae996ea37ef470ed73', 'request_id': 'req-d33545c2-deff-45db-a9cb-d9387ef4612f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:03.791491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60fc2b51ab8f46ae996ea37ef470ed73', 'request_id': 'req-d33545c2-deff-45db-a9cb-d9387ef4612f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:03.808739 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] The token cache is being invalidate because project 60fc2b51ab8f46ae996ea37ef470ed73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:03.809036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:03.809765 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-d33545c2-deff-45db-a9cb-d9387ef4612f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:03.816905 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1393/2812] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:03 2022] DELETE /identity/v3/projects/60fc2b51ab8f46ae996ea37ef470ed73 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:03.820664 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b71e6e-0dbc-4e50-b4ca-546866d622ef None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:03.821132 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b71e6e-0dbc-4e50-b4ca-546866d622ef None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:03.822122 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b71e6e-0dbc-4e50-b4ca-546866d622ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:03.842777 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-45b71e6e-0dbc-4e50-b4ca-546866d622ef None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:03.844262 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-45b71e6e-0dbc-4e50-b4ca-546866d622ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:03.914096 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1420/2813] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:09:03.915282 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703288d9-3447-4e5e-9895-d6be19ca1d3d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:03.915574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703288d9-3447-4e5e-9895-d6be19ca1d3d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:03.915857 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703288d9-3447-4e5e-9895-d6be19ca1d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:03.941166 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-703288d9-3447-4e5e-9895-d6be19ca1d3d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:03.942507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-703288d9-3447-4e5e-9895-d6be19ca1d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:03.952220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1394/2814] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:03.956312 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b71e6e-0dbc-4e50-b4ca-546866d622ef None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:03.967309 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b71e6e-0dbc-4e50-b4ca-546866d622ef None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:03.971878 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00bd4a-bcf0-4fec-ac9a-fa2b01c9248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:03.972557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00bd4a-bcf0-4fec-ac9a-fa2b01c9248b admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:03.972967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00bd4a-bcf0-4fec-ac9a-fa2b01c9248b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:03.973258 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00bd4a-bcf0-4fec-ac9a-fa2b01c9248b admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:03.974023 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00bd4a-bcf0-4fec-ac9a-fa2b01c9248b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:09:03.974447 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00bd4a-bcf0-4fec-ac9a-fa2b01c9248b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:03.976668 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00bd4a-bcf0-4fec-ac9a-fa2b01c9248b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:03.992018 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1421/2815] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 10:09:03.993309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703288d9-3447-4e5e-9895-d6be19ca1d3d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:04.002880 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703288d9-3447-4e5e-9895-d6be19ca1d3d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:04.008147 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc7ebb-60e0-4e06-b65f-2d6b5c6a0952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:04.008895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc7ebb-60e0-4e06-b65f-2d6b5c6a0952 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:04.009175 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc7ebb-60e0-4e06-b65f-2d6b5c6a0952 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:04.009451 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc7ebb-60e0-4e06-b65f-2d6b5c6a0952 admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:04.010162 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc7ebb-60e0-4e06-b65f-2d6b5c6a0952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:04.012090 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc7ebb-60e0-4e06-b65f-2d6b5c6a0952 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:04.150002 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1395/2816] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:09:03 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:04.154650 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00bd4a-bcf0-4fec-ac9a-fa2b01c9248b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:04.165344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00bd4a-bcf0-4fec-ac9a-fa2b01c9248b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:04.235261 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a3aef-38a9-496d-aea5-bea1dea5eb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:04.236233 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a3aef-38a9-496d-aea5-bea1dea5eb6f admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:04.236583 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a3aef-38a9-496d-aea5-bea1dea5eb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:04.236997 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a3aef-38a9-496d-aea5-bea1dea5eb6f admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:04.237787 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a3aef-38a9-496d-aea5-bea1dea5eb6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:04.239798 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a3aef-38a9-496d-aea5-bea1dea5eb6f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:04.371472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1422/2817] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:09:04 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:04.375720 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc7ebb-60e0-4e06-b65f-2d6b5c6a0952 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:04.386937 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc7ebb-60e0-4e06-b65f-2d6b5c6a0952 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:04.391475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c1608-29c8-4329-ac5c-9f637e083ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:04.392604 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c1608-29c8-4329-ac5c-9f637e083ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:04.392893 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c1608-29c8-4329-ac5c-9f637e083ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:04.393172 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c1608-29c8-4329-ac5c-9f637e083ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:04.393926 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c1608-29c8-4329-ac5c-9f637e083ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:04.395536 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c1608-29c8-4329-ac5c-9f637e083ab0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:04.407133 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1396/2818] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:04.411302 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a3aef-38a9-496d-aea5-bea1dea5eb6f admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:04.421534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a3aef-38a9-496d-aea5-bea1dea5eb6f admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:04.426164 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d3adc-7b04-440e-8b7d-5ebc6ecbedf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:04.426848 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3adc-7b04-440e-8b7d-5ebc6ecbedf9 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:04.427119 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3adc-7b04-440e-8b7d-5ebc6ecbedf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:04.427398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3adc-7b04-440e-8b7d-5ebc6ecbedf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:04.428125 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d3adc-7b04-440e-8b7d-5ebc6ecbedf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:04.429743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d3adc-7b04-440e-8b7d-5ebc6ecbedf9 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:04.442229 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1423/2819] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:04.447081 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c1608-29c8-4329-ac5c-9f637e083ab0 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:04.453777 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:04.457800 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c1608-29c8-4329-ac5c-9f637e083ab0 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:04.463068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3557c68-dc3e-48ea-83e7-687130745820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:04.463954 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3557c68-dc3e-48ea-83e7-687130745820 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:04.464243 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3557c68-dc3e-48ea-83e7-687130745820 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:04.464520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3557c68-dc3e-48ea-83e7-687130745820 admin admin] PATH_INFO: `/v3/projects/38dfc11b89fd469fbf6b9a9817a55e6a/users/89c06f8c515f4cd9b41556dbde128e69/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:04.479665 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3557c68-dc3e-48ea-83e7-687130745820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38dfc11b89fd469fbf6b9a9817a55e6a, user_id=89c06f8c515f4cd9b41556dbde128e69, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:04.482522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3557c68-dc3e-48ea-83e7-687130745820 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:04.599524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1397/2820] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:04 2022] PUT /identity/v3/projects/38dfc11b89fd469fbf6b9a9817a55e6a/users/89c06f8c515f4cd9b41556dbde128e69/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:09:04.604654 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee8315-3a3a-49c1-bf56-4ce744d273ff None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:04.604971 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee8315-3a3a-49c1-bf56-4ce744d273ff None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:04.605252 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee8315-3a3a-49c1-bf56-4ce744d273ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:04.622641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-a2ee8315-3a3a-49c1-bf56-4ce744d273ff None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:04.629480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a2ee8315-3a3a-49c1-bf56-4ce744d273ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:04.664393 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:04.683904 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1424/2821] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:04.693897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3557c68-dc3e-48ea-83e7-687130745820 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:04.695788 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a3557c68-dc3e-48ea-83e7-687130745820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:04.757021 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3557c68-dc3e-48ea-83e7-687130745820 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:04.760794 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0869cad-e8f6-462f-9b97-57b1e2780f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:04.761461 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0869cad-e8f6-462f-9b97-57b1e2780f10 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:04.761749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0869cad-e8f6-462f-9b97-57b1e2780f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:04.762072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0869cad-e8f6-462f-9b97-57b1e2780f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:04.771658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0869cad-e8f6-462f-9b97-57b1e2780f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:04.773148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0869cad-e8f6-462f-9b97-57b1e2780f10 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:04.783465 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1398/2822] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:05.923255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee8315-3a3a-49c1-bf56-4ce744d273ff None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:05.924690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-a2ee8315-3a3a-49c1-bf56-4ce744d273ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:05.936645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee8315-3a3a-49c1-bf56-4ce744d273ff None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:05.940136 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b48aa-b3be-4242-a757-0bcb7bccdf4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:05.942255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b48aa-b3be-4242-a757-0bcb7bccdf4a service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:05.942255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b48aa-b3be-4242-a757-0bcb7bccdf4a service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:05.942255 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b48aa-b3be-4242-a757-0bcb7bccdf4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:05.943690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-e60b48aa-b3be-4242-a757-0bcb7bccdf4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:05.955211 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b48aa-b3be-4242-a757-0bcb7bccdf4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:05.956614 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b48aa-b3be-4242-a757-0bcb7bccdf4a service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:05.966785 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1425/2823] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:06.528421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bbd33-11a8-4305-872b-e6c8603d385c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:06.529023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bbd33-11a8-4305-872b-e6c8603d385c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:06.529023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bbd33-11a8-4305-872b-e6c8603d385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:06.571503 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-485bbd33-11a8-4305-872b-e6c8603d385c None None] MFA Rules not processed for user `89c06f8c515f4cd9b41556dbde128e69`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:06.572855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-485bbd33-11a8-4305-872b-e6c8603d385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:06.648359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1399/2824] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 10:09:06.652915 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b48aa-b3be-4242-a757-0bcb7bccdf4a service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:06.663281 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b48aa-b3be-4242-a757-0bcb7bccdf4a service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:06.668024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745811e3-01fc-45bb-8da5-b85d4021c647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:06.668836 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745811e3-01fc-45bb-8da5-b85d4021c647 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:06.669123 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745811e3-01fc-45bb-8da5-b85d4021c647 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:06.669402 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745811e3-01fc-45bb-8da5-b85d4021c647 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:06.670127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745811e3-01fc-45bb-8da5-b85d4021c647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:06.672020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745811e3-01fc-45bb-8da5-b85d4021c647 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:06.767491 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1426/2825] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:09:06 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:06.771611 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bbd33-11a8-4305-872b-e6c8603d385c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:06.782525 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bbd33-11a8-4305-872b-e6c8603d385c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:06.842658 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb745fda-1b14-4d6c-9372-cfc29b857a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:06.843253 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb745fda-1b14-4d6c-9372-cfc29b857a06 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:06.843539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb745fda-1b14-4d6c-9372-cfc29b857a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:06.843820 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb745fda-1b14-4d6c-9372-cfc29b857a06 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:06.844520 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb745fda-1b14-4d6c-9372-cfc29b857a06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:06.846449 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb745fda-1b14-4d6c-9372-cfc29b857a06 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:06.928018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1400/2826] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:09:06 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:06.929304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745811e3-01fc-45bb-8da5-b85d4021c647 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:06.939710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745811e3-01fc-45bb-8da5-b85d4021c647 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:06.944304 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff15a2b-b20b-4402-b257-2135c712c65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:06.944954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff15a2b-b20b-4402-b257-2135c712c65c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:06.945237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff15a2b-b20b-4402-b257-2135c712c65c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:06.945512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff15a2b-b20b-4402-b257-2135c712c65c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:06.946269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff15a2b-b20b-4402-b257-2135c712c65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:06.947895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff15a2b-b20b-4402-b257-2135c712c65c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:06.960828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1427/2827] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:06.965167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb745fda-1b14-4d6c-9372-cfc29b857a06 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:06.970421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:06.976537 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb745fda-1b14-4d6c-9372-cfc29b857a06 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:06.983143 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb316f-c4e1-49cb-932e-ce2dcf93d4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:06.984383 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb316f-c4e1-49cb-932e-ce2dcf93d4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:06.984772 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb316f-c4e1-49cb-932e-ce2dcf93d4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:06.985128 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb316f-c4e1-49cb-932e-ce2dcf93d4cb admin admin] PATH_INFO: `/v3/projects/1641f17cf6d84281b9d8292cd7533263/users/1522011b801643c5ac6b6f011a690e5b/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:07.009681 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb316f-c4e1-49cb-932e-ce2dcf93d4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1641f17cf6d84281b9d8292cd7533263, user_id=1522011b801643c5ac6b6f011a690e5b, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:07.021761 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb316f-c4e1-49cb-932e-ce2dcf93d4cb admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:07.126567 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1401/2828] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:06 2022] PUT /identity/v3/projects/1641f17cf6d84281b9d8292cd7533263/users/1522011b801643c5ac6b6f011a690e5b/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:07.130962 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff15a2b-b20b-4402-b257-2135c712c65c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:07.144799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff15a2b-b20b-4402-b257-2135c712c65c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:07.248493 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085ee8-4af5-4563-a64a-031f4442f75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:07.249362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085ee8-4af5-4563-a64a-031f4442f75c admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:07.249714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085ee8-4af5-4563-a64a-031f4442f75c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:07.250124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085ee8-4af5-4563-a64a-031f4442f75c admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:07.250965 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81085ee8-4af5-4563-a64a-031f4442f75c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:07.252789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81085ee8-4af5-4563-a64a-031f4442f75c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:07.264933 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1428/2829] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:09:07.268995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb316f-c4e1-49cb-932e-ce2dcf93d4cb admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:07.280005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb316f-c4e1-49cb-932e-ce2dcf93d4cb admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:07.284607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493b798-dd2d-480c-bc73-f6257efcf73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:07.285487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493b798-dd2d-480c-bc73-f6257efcf73c admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:07.285770 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493b798-dd2d-480c-bc73-f6257efcf73c admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:07.286079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493b798-dd2d-480c-bc73-f6257efcf73c admin admin] PATH_INFO: `/v3/projects/1641f17cf6d84281b9d8292cd7533263/users/1522011b801643c5ac6b6f011a690e5b/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:07.288064 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493b798-dd2d-480c-bc73-f6257efcf73c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1641f17cf6d84281b9d8292cd7533263, user_id=1522011b801643c5ac6b6f011a690e5b, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:07.290760 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493b798-dd2d-480c-bc73-f6257efcf73c admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:07.301411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1402/2830] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:07 2022] PUT /identity/v3/projects/1641f17cf6d84281b9d8292cd7533263/users/1522011b801643c5ac6b6f011a690e5b/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:08.906368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea0ca0-17b0-43ac-91ad-6162e6717183 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:08.906862 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea0ca0-17b0-43ac-91ad-6162e6717183 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:08.906967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea0ca0-17b0-43ac-91ad-6162e6717183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:08.952902 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-56ea0ca0-17b0-43ac-91ad-6162e6717183 None None] MFA Rules not processed for user `1522011b801643c5ac6b6f011a690e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:08.955714 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-56ea0ca0-17b0-43ac-91ad-6162e6717183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:09.057404 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1429/2831] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:09.069807 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493b798-dd2d-480c-bc73-f6257efcf73c admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:09.071683 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-d493b798-dd2d-480c-bc73-f6257efcf73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:09.142894 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493b798-dd2d-480c-bc73-f6257efcf73c admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:09.152947 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650ea26-bb0f-4767-82bd-5200fc3dadd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:09.153731 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650ea26-bb0f-4767-82bd-5200fc3dadd5 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:09.154124 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650ea26-bb0f-4767-82bd-5200fc3dadd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:09.154475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650ea26-bb0f-4767-82bd-5200fc3dadd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:09.212111 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650ea26-bb0f-4767-82bd-5200fc3dadd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:09.213459 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650ea26-bb0f-4767-82bd-5200fc3dadd5 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:09.222501 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1403/2832] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:11.073014 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea0ca0-17b0-43ac-91ad-6162e6717183 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:11.074542 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-56ea0ca0-17b0-43ac-91ad-6162e6717183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:11.089335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea0ca0-17b0-43ac-91ad-6162e6717183 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:11.092891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34a120-161b-469f-9d5a-93a65d913a3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:11.093621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34a120-161b-469f-9d5a-93a65d913a3e service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:11.093943 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34a120-161b-469f-9d5a-93a65d913a3e service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:11.094222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34a120-161b-469f-9d5a-93a65d913a3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:11.105156 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34a120-161b-469f-9d5a-93a65d913a3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:11.106726 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34a120-161b-469f-9d5a-93a65d913a3e service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:11.121805 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1430/2833] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:13.709154 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650ea26-bb0f-4767-82bd-5200fc3dadd5 service cinder] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:13.711217 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2650ea26-bb0f-4767-82bd-5200fc3dadd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:13.780491 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650ea26-bb0f-4767-82bd-5200fc3dadd5 service cinder] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:13.785130 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:13.786041 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:13.786337 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:13.787223 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] PATH_INFO: `/v3/users/646fecf4405c41a0bc062c6b54fd311c` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:13.788373 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=646fecf4405c41a0bc062c6b54fd311c)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:13.792095 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:13.813624 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:14.161031 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646fecf4405c41a0bc062c6b54fd311c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.161612 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646fecf4405c41a0bc062c6b54fd311c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.173079 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646fecf4405c41a0bc062c6b54fd311c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.179953 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646fecf4405c41a0bc062c6b54fd311c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.336313 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646fecf4405c41a0bc062c6b54fd311c'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.342631 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1404/2834] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:13 2022] DELETE /identity/v3/users/646fecf4405c41a0bc062c6b54fd311c => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:14.573386 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34a120-161b-469f-9d5a-93a65d913a3e service cinder] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:14.585169 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34a120-161b-469f-9d5a-93a65d913a3e service cinder] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:14.589086 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:14.645353 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:14.646260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:14.646549 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:14.646831 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] PATH_INFO: `/v3/projects/5476f21b29944eb69e32a2d97b328aed` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:14.648040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5476f21b29944eb69e32a2d97b328aed)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:14.649967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:14.782389 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5476f21b29944eb69e32a2d97b328aed', 'request_id': 'req-3dcd1373-08e7-4672-bd66-65bd315c2d31'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.788222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5476f21b29944eb69e32a2d97b328aed', 'request_id': 'req-3dcd1373-08e7-4672-bd66-65bd315c2d31'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.867467 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5476f21b29944eb69e32a2d97b328aed', 'request_id': 'req-3dcd1373-08e7-4672-bd66-65bd315c2d31'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.882347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] The token cache is being invalidate because project 5476f21b29944eb69e32a2d97b328aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:14.882644 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.883327 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:14.885094 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1431/2835] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:14 2022] DELETE /identity/v3/projects/5476f21b29944eb69e32a2d97b328aed => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:14.889072 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:14.890675 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:14.904036 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338151f-1f50-4720-9207-bb1509ebf2d3 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:14.968328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:14.971106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:14.971106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:14.971106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] PATH_INFO: `/v3/users/6b55eed0af3e4b838d62750fee9cbe98` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:14.971106 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b55eed0af3e4b838d62750fee9cbe98)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:14.973086 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:15.332082 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b55eed0af3e4b838d62750fee9cbe98'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:15.332742 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b55eed0af3e4b838d62750fee9cbe98'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:15.344831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b55eed0af3e4b838d62750fee9cbe98'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:15.352360 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b55eed0af3e4b838d62750fee9cbe98'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:15.421359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b55eed0af3e4b838d62750fee9cbe98'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:15.427552 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1405/2836] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:14 2022] DELETE /identity/v3/users/6b55eed0af3e4b838d62750fee9cbe98 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:15.680410 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:15.681964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:15.696206 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd1373-08e7-4672-bd66-65bd315c2d31 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:15.761977 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:15.762854 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:15.763140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:15.763417 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] PATH_INFO: `/v3/projects/587f4a774d5e46d1a61cbe83c1d42631` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:15.764600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=587f4a774d5e46d1a61cbe83c1d42631)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:15.766559 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:15.886104 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:15.892666 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587f4a774d5e46d1a61cbe83c1d42631', 'request_id': 'req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:15.898568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587f4a774d5e46d1a61cbe83c1d42631', 'request_id': 'req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:16.005010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587f4a774d5e46d1a61cbe83c1d42631', 'request_id': 'req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:16.017315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] The token cache is being invalidate because project 587f4a774d5e46d1a61cbe83c1d42631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:16.018160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:16.018287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:16.019799 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1432/2837] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:15 2022] DELETE /identity/v3/projects/587f4a774d5e46d1a61cbe83c1d42631 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:25.423873 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=17,keystone:SELECT=73,keystone:INSERT=2 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:26.673443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:27.782195 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:27.783653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:27.797314 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fcf4c-0919-4fc9-8c6b-57f2a32669ca admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:27.853186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:27.854084 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:27.854371 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:27.854648 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] PATH_INFO: `/v3/users/89c06f8c515f4cd9b41556dbde128e69` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:27.855814 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c06f8c515f4cd9b41556dbde128e69)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:27.857735 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:28.022941 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c06f8c515f4cd9b41556dbde128e69'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.023603 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c06f8c515f4cd9b41556dbde128e69'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.036074 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c06f8c515f4cd9b41556dbde128e69'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.043633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c06f8c515f4cd9b41556dbde128e69'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.101103 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c06f8c515f4cd9b41556dbde128e69'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.107911 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1406/2838] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:27 2022] DELETE /identity/v3/users/89c06f8c515f4cd9b41556dbde128e69 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:28.331648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:28.335335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:28.346360 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ca4e-c0aa-4e0d-9709-e390371b3cbf admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:28.404029 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:28.404916 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:28.405201 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:28.405482 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] PATH_INFO: `/v3/projects/38dfc11b89fd469fbf6b9a9817a55e6a` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:28.406695 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38dfc11b89fd469fbf6b9a9817a55e6a)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:28.408608 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:28.528607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38dfc11b89fd469fbf6b9a9817a55e6a', 'request_id': 'req-4eeebb12-baa6-433f-93b5-e86ea4f5305d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.534113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38dfc11b89fd469fbf6b9a9817a55e6a', 'request_id': 'req-4eeebb12-baa6-433f-93b5-e86ea4f5305d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.608895 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38dfc11b89fd469fbf6b9a9817a55e6a', 'request_id': 'req-4eeebb12-baa6-433f-93b5-e86ea4f5305d'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.620919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] The token cache is being invalidate because project 38dfc11b89fd469fbf6b9a9817a55e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:28.621203 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.624091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.624091 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1433/2839] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:28 2022] DELETE /identity/v3/projects/38dfc11b89fd469fbf6b9a9817a55e6a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:28.627476 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:28.629176 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:28.642400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f39495-3265-49fc-8cbe-e6606c4ec054 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:28.696673 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:28.698186 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:28.698474 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:28.698749 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] PATH_INFO: `/v3/users/1522011b801643c5ac6b6f011a690e5b` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:28.699918 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1522011b801643c5ac6b6f011a690e5b)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:28.701846 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:28.724590 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:28.943171 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1522011b801643c5ac6b6f011a690e5b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.943811 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1522011b801643c5ac6b6f011a690e5b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.956539 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1522011b801643c5ac6b6f011a690e5b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:28.964200 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1522011b801643c5ac6b6f011a690e5b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:29.061267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4e017020-2116-4299-bfaa-c4775b707016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1522011b801643c5ac6b6f011a690e5b'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:29.067734 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1407/2840] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:28 2022] DELETE /identity/v3/users/1522011b801643c5ac6b6f011a690e5b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:29.302529 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:29.303955 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:29.318939 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeebb12-baa6-433f-93b5-e86ea4f5305d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:29.377409 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:29.378560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:29.378844 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:29.379120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] PATH_INFO: `/v3/projects/1641f17cf6d84281b9d8292cd7533263` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:29.380331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=1641f17cf6d84281b9d8292cd7533263)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:29.382266 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:29.478374 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:29.490033 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1641f17cf6d84281b9d8292cd7533263', 'request_id': 'req-0c79b491-9b07-409e-a463-17f0a27131ac'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:29.496280 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1641f17cf6d84281b9d8292cd7533263', 'request_id': 'req-0c79b491-9b07-409e-a463-17f0a27131ac'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:29.569856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1641f17cf6d84281b9d8292cd7533263', 'request_id': 'req-0c79b491-9b07-409e-a463-17f0a27131ac'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:29.581497 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] The token cache is being invalidate because project 1641f17cf6d84281b9d8292cd7533263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:29.582961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:29.582961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-0c79b491-9b07-409e-a463-17f0a27131ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:29.584030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1434/2841] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:29 2022] DELETE /identity/v3/projects/1641f17cf6d84281b9d8292cd7533263 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:29.589134 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287a917-6f22-4706-a126-fb2febd3351d None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:29.589422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287a917-6f22-4706-a126-fb2febd3351d None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:29.589696 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287a917-6f22-4706-a126-fb2febd3351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:29.607727 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-2287a917-6f22-4706-a126-fb2febd3351d None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:29.608916 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-2287a917-6f22-4706-a126-fb2febd3351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:29.668912 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1408/2842] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:29.677003 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44fea2-16ca-40d3-95ea-ce8f4453ccbd None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:29.677299 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44fea2-16ca-40d3-95ea-ce8f4453ccbd None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:29.680455 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44fea2-16ca-40d3-95ea-ce8f4453ccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:29.696209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-0e44fea2-16ca-40d3-95ea-ce8f4453ccbd None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:29.697437 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-0e44fea2-16ca-40d3-95ea-ce8f4453ccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:29.705905 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1435/2843] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:29.710101 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287a917-6f22-4706-a126-fb2febd3351d None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:29.720021 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287a917-6f22-4706-a126-fb2febd3351d None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:29.724663 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881fd6f-4df6-42d2-a9f7-ba05417dd732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:29.725319 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881fd6f-4df6-42d2-a9f7-ba05417dd732 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:29.725608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881fd6f-4df6-42d2-a9f7-ba05417dd732 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:29.725914 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881fd6f-4df6-42d2-a9f7-ba05417dd732 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:29.726637 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881fd6f-4df6-42d2-a9f7-ba05417dd732 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:09:29.726949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881fd6f-4df6-42d2-a9f7-ba05417dd732 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:29.729050 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881fd6f-4df6-42d2-a9f7-ba05417dd732 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:29.740422 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1409/2844] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:09:29.745350 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44fea2-16ca-40d3-95ea-ce8f4453ccbd None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:29.754873 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44fea2-16ca-40d3-95ea-ce8f4453ccbd None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:29.759507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d514b8b-84b7-4f06-b050-e3ccb577992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:29.760183 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d514b8b-84b7-4f06-b050-e3ccb577992c admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:29.760472 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d514b8b-84b7-4f06-b050-e3ccb577992c admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:29.760752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d514b8b-84b7-4f06-b050-e3ccb577992c admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:29.761481 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d514b8b-84b7-4f06-b050-e3ccb577992c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:29.763465 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d514b8b-84b7-4f06-b050-e3ccb577992c admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:29.843319 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1436/2845] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:09:29 2022] POST /identity/v3/projects => generated 361 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:09:29.845844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881fd6f-4df6-42d2-a9f7-ba05417dd732 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:29.856513 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881fd6f-4df6-42d2-a9f7-ba05417dd732 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:29.911805 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1e8f5-22df-43bf-a6d4-71bd6c223da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:29.912467 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1e8f5-22df-43bf-a6d4-71bd6c223da5 admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:29.912756 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1e8f5-22df-43bf-a6d4-71bd6c223da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:29.913051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1e8f5-22df-43bf-a6d4-71bd6c223da5 admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:29.913721 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1e8f5-22df-43bf-a6d4-71bd6c223da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:29.915724 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1e8f5-22df-43bf-a6d4-71bd6c223da5 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:30.003408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1410/2846] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:09:29 2022] POST /identity/v3/users => generated 281 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:30.007450 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d514b8b-84b7-4f06-b050-e3ccb577992c admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:30.018020 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d514b8b-84b7-4f06-b050-e3ccb577992c admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:30.022641 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ba3ec-4c3f-47e7-b26d-a1489a64986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:30.023295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ba3ec-4c3f-47e7-b26d-a1489a64986e admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:30.023578 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ba3ec-4c3f-47e7-b26d-a1489a64986e admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:30.023849 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ba3ec-4c3f-47e7-b26d-a1489a64986e admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:30.024565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ba3ec-4c3f-47e7-b26d-a1489a64986e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:30.026710 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ba3ec-4c3f-47e7-b26d-a1489a64986e admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:30.036954 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1437/2847] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:30.040762 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1e8f5-22df-43bf-a6d4-71bd6c223da5 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:30.051316 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1e8f5-22df-43bf-a6d4-71bd6c223da5 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:30.055957 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4123f3-b401-4db3-85e8-f39f956303e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:30.056608 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4123f3-b401-4db3-85e8-f39f956303e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:30.056888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4123f3-b401-4db3-85e8-f39f956303e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:30.057165 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4123f3-b401-4db3-85e8-f39f956303e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:30.057903 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4123f3-b401-4db3-85e8-f39f956303e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:30.059504 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4123f3-b401-4db3-85e8-f39f956303e2 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:30.069664 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1411/2848] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:30.073572 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ba3ec-4c3f-47e7-b26d-a1489a64986e admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:30.083536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ba3ec-4c3f-47e7-b26d-a1489a64986e admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:30.088826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06ba8c-857a-4e04-9e73-51f3274e40f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:30.089699 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06ba8c-857a-4e04-9e73-51f3274e40f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:30.090030 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06ba8c-857a-4e04-9e73-51f3274e40f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:30.090315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06ba8c-857a-4e04-9e73-51f3274e40f6 admin admin] PATH_INFO: `/v3/projects/6560b66929bf4103a56f87165cd3d24b/users/38da398fbdbf402b8d10c221e39b5445/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:30.105688 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06ba8c-857a-4e04-9e73-51f3274e40f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6560b66929bf4103a56f87165cd3d24b, user_id=38da398fbdbf402b8d10c221e39b5445, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:30.108315 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06ba8c-857a-4e04-9e73-51f3274e40f6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:30.244584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1438/2849] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:30 2022] PUT /identity/v3/projects/6560b66929bf4103a56f87165cd3d24b/users/38da398fbdbf402b8d10c221e39b5445/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:30.304251 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878acef-9e69-42ff-a076-53a58bc7765c None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:30.304548 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878acef-9e69-42ff-a076-53a58bc7765c None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:30.304831 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878acef-9e69-42ff-a076-53a58bc7765c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:30.311615 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:30.330687 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-1878acef-9e69-42ff-a076-53a58bc7765c None None] MFA Rules not processed for user `38da398fbdbf402b8d10c221e39b5445`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:30.331895 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1878acef-9e69-42ff-a076-53a58bc7765c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:30.395574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1412/2850] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:30.432593 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06ba8c-857a-4e04-9e73-51f3274e40f6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:30.437589 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-ae06ba8c-857a-4e04-9e73-51f3274e40f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:30.490711 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06ba8c-857a-4e04-9e73-51f3274e40f6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:30.494443 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d9ce3-a3e0-420b-a6e8-83c776cc4c16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:30.495084 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d9ce3-a3e0-420b-a6e8-83c776cc4c16 service cinder] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:30.495372 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d9ce3-a3e0-420b-a6e8-83c776cc4c16 service cinder] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:30.495655 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d9ce3-a3e0-420b-a6e8-83c776cc4c16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:30.504139 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d9ce3-a3e0-420b-a6e8-83c776cc4c16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:30.505607 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489d9ce3-a3e0-420b-a6e8-83c776cc4c16 service cinder] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:30.514967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1439/2851] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 10:09:30.529311 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878acef-9e69-42ff-a076-53a58bc7765c None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:30.590443 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878acef-9e69-42ff-a076-53a58bc7765c None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:30.594971 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:30.595833 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:30.596114 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:30.596391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] PATH_INFO: `/v3/users/38da398fbdbf402b8d10c221e39b5445` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:30.597559 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=38da398fbdbf402b8d10c221e39b5445)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:30.599524 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:30.799381 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38da398fbdbf402b8d10c221e39b5445'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:30.800005 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38da398fbdbf402b8d10c221e39b5445'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:30.812670 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38da398fbdbf402b8d10c221e39b5445'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:30.819432 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38da398fbdbf402b8d10c221e39b5445'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:30.899888 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38da398fbdbf402b8d10c221e39b5445'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:30.906024 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1413/2852] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:30 2022] DELETE /identity/v3/users/38da398fbdbf402b8d10c221e39b5445 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:30.910724 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a8083-1298-4e7b-9ff2-65648f19f43f None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:30.918103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a8083-1298-4e7b-9ff2-65648f19f43f None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:30.918103 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a8083-1298-4e7b-9ff2-65648f19f43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:30.928191 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-c02a8083-1298-4e7b-9ff2-65648f19f43f None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:30.929435 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c02a8083-1298-4e7b-9ff2-65648f19f43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:30.981207 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:30.989347 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1440/2853] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:30.999007 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:31.000293 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:31.014959 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:31.057155 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57babf3-2f73-4b8b-b932-bc9a47eb29cd admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:31.060942 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97349b-c8a9-4c9b-a813-a8eb01b6fce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:31.061588 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97349b-c8a9-4c9b-a813-a8eb01b6fce7 service neutron] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:31.061876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97349b-c8a9-4c9b-a813-a8eb01b6fce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:31.062180 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97349b-c8a9-4c9b-a813-a8eb01b6fce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:31.071157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97349b-c8a9-4c9b-a813-a8eb01b6fce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:31.072591 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97349b-c8a9-4c9b-a813-a8eb01b6fce7 service neutron] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:31.082494 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1414/2854] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:31.549114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a8083-1298-4e7b-9ff2-65648f19f43f None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:31.550742 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-c02a8083-1298-4e7b-9ff2-65648f19f43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:31.563326 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a8083-1298-4e7b-9ff2-65648f19f43f None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:31.568045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:31.568934 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:31.569221 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:31.569502 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] PATH_INFO: `/v3/projects/6560b66929bf4103a56f87165cd3d24b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:31.570754 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6560b66929bf4103a56f87165cd3d24b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:31.573277 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:31.754504 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6560b66929bf4103a56f87165cd3d24b', 'request_id': 'req-6343cb53-30bf-4811-840b-5308740e7f00'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:31.760818 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6560b66929bf4103a56f87165cd3d24b', 'request_id': 'req-6343cb53-30bf-4811-840b-5308740e7f00'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:31.860269 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6560b66929bf4103a56f87165cd3d24b', 'request_id': 'req-6343cb53-30bf-4811-840b-5308740e7f00'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:31.872308 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] The token cache is being invalidate because project 6560b66929bf4103a56f87165cd3d24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87026) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:31.872596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:31.873268 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6343cb53-30bf-4811-840b-5308740e7f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:31.874919 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1441/2855] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:31 2022] DELETE /identity/v3/projects/6560b66929bf4103a56f87165cd3d24b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:31.879975 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2044eb2-8a56-45fb-bb6a-2d06c7c2500e None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:31.880267 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2044eb2-8a56-45fb-bb6a-2d06c7c2500e None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:31.880546 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2044eb2-8a56-45fb-bb6a-2d06c7c2500e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:31.897690 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-a2044eb2-8a56-45fb-bb6a-2d06c7c2500e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:31.898952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-a2044eb2-8a56-45fb-bb6a-2d06c7c2500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:31.957023 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1415/2856] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:31.964828 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179ce8-74e4-437f-8508-e785fa26be4e None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:31.965124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179ce8-74e4-437f-8508-e785fa26be4e None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:31.965411 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179ce8-74e4-437f-8508-e785fa26be4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:31.983817 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-57179ce8-74e4-437f-8508-e785fa26be4e None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:31.985016 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-57179ce8-74e4-437f-8508-e785fa26be4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:31.992961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1442/2857] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:32.000400 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2044eb2-8a56-45fb-bb6a-2d06c7c2500e None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:32.010250 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2044eb2-8a56-45fb-bb6a-2d06c7c2500e None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:32.014795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbdfbd-cd68-4dae-a894-852bf5dbf410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:32.015543 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbdfbd-cd68-4dae-a894-852bf5dbf410 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:32.016088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbdfbd-cd68-4dae-a894-852bf5dbf410 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:32.016088 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbdfbd-cd68-4dae-a894-852bf5dbf410 admin admin] PATH_INFO: `/v3/domains` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:32.016818 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbdfbd-cd68-4dae-a894-852bf5dbf410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87025) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:09:32.017123 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbdfbd-cd68-4dae-a894-852bf5dbf410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:32.018753 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbdfbd-cd68-4dae-a894-852bf5dbf410 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:32.030222 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1416/2858] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:09:32.035114 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179ce8-74e4-437f-8508-e785fa26be4e None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:32.044839 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179ce8-74e4-437f-8508-e785fa26be4e None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:32.049555 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f67dd-7a1e-4168-bb98-9126f98ba8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:32.050249 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f67dd-7a1e-4168-bb98-9126f98ba8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:32.050538 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f67dd-7a1e-4168-bb98-9126f98ba8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:32.050819 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f67dd-7a1e-4168-bb98-9126f98ba8b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:32.051514 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f67dd-7a1e-4168-bb98-9126f98ba8b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:32.053405 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f67dd-7a1e-4168-bb98-9126f98ba8b7 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:32.153958 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1443/2859] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:09:32 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 10:09:32.158235 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbdfbd-cd68-4dae-a894-852bf5dbf410 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:32.168560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbdfbd-cd68-4dae-a894-852bf5dbf410 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:32.223429 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c973bc9-6525-44a9-85c8-095c99d8679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:32.224092 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c973bc9-6525-44a9-85c8-095c99d8679a admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:32.224378 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c973bc9-6525-44a9-85c8-095c99d8679a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:32.224657 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c973bc9-6525-44a9-85c8-095c99d8679a admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:32.225339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c973bc9-6525-44a9-85c8-095c99d8679a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:32.227286 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c973bc9-6525-44a9-85c8-095c99d8679a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:32.307220 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1417/2860] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:09:32 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:32.311600 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f67dd-7a1e-4168-bb98-9126f98ba8b7 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:32.321820 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f67dd-7a1e-4168-bb98-9126f98ba8b7 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:32.327621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46690e3a-6bc2-45a6-8daa-23f44ddfc98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:32.327621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46690e3a-6bc2-45a6-8daa-23f44ddfc98d admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:32.327621 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46690e3a-6bc2-45a6-8daa-23f44ddfc98d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:32.327992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46690e3a-6bc2-45a6-8daa-23f44ddfc98d admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:32.328376 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46690e3a-6bc2-45a6-8daa-23f44ddfc98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:32.329992 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46690e3a-6bc2-45a6-8daa-23f44ddfc98d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:32.340804 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1444/2861] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 10:09:32.344653 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c973bc9-6525-44a9-85c8-095c99d8679a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:32.355228 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c973bc9-6525-44a9-85c8-095c99d8679a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:32.359845 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253b580-cc94-4bb7-9899-aa3dd5b425ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:32.360493 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b580-cc94-4bb7-9899-aa3dd5b425ff admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:32.360781 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b580-cc94-4bb7-9899-aa3dd5b425ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:32.361063 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253b580-cc94-4bb7-9899-aa3dd5b425ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:32.361779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253b580-cc94-4bb7-9899-aa3dd5b425ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:32.363408 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253b580-cc94-4bb7-9899-aa3dd5b425ff admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:32.373238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1418/2862] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:32.377228 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46690e3a-6bc2-45a6-8daa-23f44ddfc98d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:32.387311 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46690e3a-6bc2-45a6-8daa-23f44ddfc98d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:32.391886 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c48e73-843e-4c00-ad50-8ce7657059a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:32.392789 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c48e73-843e-4c00-ad50-8ce7657059a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:32.393102 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c48e73-843e-4c00-ad50-8ce7657059a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:32.393390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c48e73-843e-4c00-ad50-8ce7657059a6 admin admin] PATH_INFO: `/v3/projects/ab2daf36d91549abae621d21f5f05689/users/bd97191264094d37aafd84daade95a8b/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:32.408524 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c48e73-843e-4c00-ad50-8ce7657059a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2daf36d91549abae621d21f5f05689, user_id=bd97191264094d37aafd84daade95a8b, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:32.411214 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c48e73-843e-4c00-ad50-8ce7657059a6 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:32.512648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1445/2863] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:32 2022] PUT /identity/v3/projects/ab2daf36d91549abae621d21f5f05689/users/bd97191264094d37aafd84daade95a8b/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:32.522708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72babeab-4f6a-437d-8401-2870b82953db None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:32.523109 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72babeab-4f6a-437d-8401-2870b82953db None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:32.523393 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72babeab-4f6a-437d-8401-2870b82953db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:32.551188 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-72babeab-4f6a-437d-8401-2870b82953db None None] MFA Rules not processed for user `bd97191264094d37aafd84daade95a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:32.552405 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-72babeab-4f6a-437d-8401-2870b82953db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:32.561426 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:32.615655 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1419/2864] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:32.620179 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c48e73-843e-4c00-ad50-8ce7657059a6 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:32.680166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c48e73-843e-4c00-ad50-8ce7657059a6 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:32.684720 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f97cd9-4824-4d7d-82df-52a25480353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:32.685370 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f97cd9-4824-4d7d-82df-52a25480353b admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:32.685657 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f97cd9-4824-4d7d-82df-52a25480353b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:32.685963 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f97cd9-4824-4d7d-82df-52a25480353b admin admin] PATH_INFO: `/v3/projects` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:32.686645 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f97cd9-4824-4d7d-82df-52a25480353b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:32.688551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f97cd9-4824-4d7d-82df-52a25480353b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:32.811978 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1446/2865] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:09:32 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:32.816249 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72babeab-4f6a-437d-8401-2870b82953db None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:32.827625 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72babeab-4f6a-437d-8401-2870b82953db None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:32.883522 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892d1d9-6870-49cd-8f07-7bfe5bec1b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:32.884191 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892d1d9-6870-49cd-8f07-7bfe5bec1b5f admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:32.884475 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892d1d9-6870-49cd-8f07-7bfe5bec1b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:32.885279 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892d1d9-6870-49cd-8f07-7bfe5bec1b5f admin admin] PATH_INFO: `/v3/users` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:32.885996 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892d1d9-6870-49cd-8f07-7bfe5bec1b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:32.887981 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892d1d9-6870-49cd-8f07-7bfe5bec1b5f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:32.997966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1420/2866] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:09:32 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:33.002043 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f97cd9-4824-4d7d-82df-52a25480353b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:33.008944 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:33.013335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f97cd9-4824-4d7d-82df-52a25480353b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:33.017973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85b66e-1858-4f57-a5eb-62cf074efaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:33.018630 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85b66e-1858-4f57-a5eb-62cf074efaff admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:33.018941 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85b66e-1858-4f57-a5eb-62cf074efaff admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:33.019246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a85b66e-1858-4f57-a5eb-62cf074efaff admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:33.019973 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85b66e-1858-4f57-a5eb-62cf074efaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:33.021603 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a85b66e-1858-4f57-a5eb-62cf074efaff admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:33.033335 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1447/2867] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:33.037340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892d1d9-6870-49cd-8f07-7bfe5bec1b5f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:33.047415 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892d1d9-6870-49cd-8f07-7bfe5bec1b5f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:33.052018 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8398bb-eaba-41c9-9a49-7b7362e0c335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:33.052885 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8398bb-eaba-41c9-9a49-7b7362e0c335 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:33.053167 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8398bb-eaba-41c9-9a49-7b7362e0c335 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:33.053448 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8398bb-eaba-41c9-9a49-7b7362e0c335 admin admin] PATH_INFO: `/v3/projects/b83034b270224c338c0186374d40f73e/users/3d9d2a23df6d463fb06cbc89df586b42/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:33.069452 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8398bb-eaba-41c9-9a49-7b7362e0c335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83034b270224c338c0186374d40f73e, user_id=3d9d2a23df6d463fb06cbc89df586b42, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:33.072750 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8398bb-eaba-41c9-9a49-7b7362e0c335 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:33.130345 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1421/2868] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:33 2022] PUT /identity/v3/projects/b83034b270224c338c0186374d40f73e/users/3d9d2a23df6d463fb06cbc89df586b42/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 10:09:33.134321 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85b66e-1858-4f57-a5eb-62cf074efaff admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:33.144690 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a85b66e-1858-4f57-a5eb-62cf074efaff admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:33.201560 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418dbe9-4daf-49b9-8184-627639c3e041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:33.202246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418dbe9-4daf-49b9-8184-627639c3e041 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:33.202540 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418dbe9-4daf-49b9-8184-627639c3e041 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:33.202824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418dbe9-4daf-49b9-8184-627639c3e041 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:33.203549 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418dbe9-4daf-49b9-8184-627639c3e041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:33.205133 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418dbe9-4daf-49b9-8184-627639c3e041 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:33.214694 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1448/2869] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:33.218795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8398bb-eaba-41c9-9a49-7b7362e0c335 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:33.229022 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8398bb-eaba-41c9-9a49-7b7362e0c335 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:33.233622 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fcb3e-dba1-4f09-ae66-0b303fda6a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:33.234563 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fcb3e-dba1-4f09-ae66-0b303fda6a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:33.234849 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fcb3e-dba1-4f09-ae66-0b303fda6a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:33.235129 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fcb3e-dba1-4f09-ae66-0b303fda6a80 admin admin] PATH_INFO: `/v3/projects/b83034b270224c338c0186374d40f73e/users/3d9d2a23df6d463fb06cbc89df586b42/roles/fe4d636f5c4842f18569347911e6cdec` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:33.237644 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fcb3e-dba1-4f09-ae66-0b303fda6a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83034b270224c338c0186374d40f73e, user_id=3d9d2a23df6d463fb06cbc89df586b42, role_id=fe4d636f5c4842f18569347911e6cdec)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:33.240317 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fcb3e-dba1-4f09-ae66-0b303fda6a80 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:33.250897 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1422/2870] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:33 2022] PUT /identity/v3/projects/b83034b270224c338c0186374d40f73e/users/3d9d2a23df6d463fb06cbc89df586b42/roles/fe4d636f5c4842f18569347911e6cdec => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:33.262752 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc232f0-ecd7-4e72-bf3b-9f23caa7df37 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:33.263047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc232f0-ecd7-4e72-bf3b-9f23caa7df37 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:33.263331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc232f0-ecd7-4e72-bf3b-9f23caa7df37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:33.288974 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-2fc232f0-ecd7-4e72-bf3b-9f23caa7df37 None None] MFA Rules not processed for user `3d9d2a23df6d463fb06cbc89df586b42`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:33.290222 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-2fc232f0-ecd7-4e72-bf3b-9f23caa7df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:33.356112 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1449/2871] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:33.367597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fcb3e-dba1-4f09-ae66-0b303fda6a80 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:33.369966 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-482fcb3e-dba1-4f09-ae66-0b303fda6a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:33.426810 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fcb3e-dba1-4f09-ae66-0b303fda6a80 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:33.430403 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8eed1c-a59f-4c96-8c77-08eba2ad9292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:33.431029 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8eed1c-a59f-4c96-8c77-08eba2ad9292 service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:33.431309 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8eed1c-a59f-4c96-8c77-08eba2ad9292 service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:33.431605 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8eed1c-a59f-4c96-8c77-08eba2ad9292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:33.447949 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:33.489460 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8eed1c-a59f-4c96-8c77-08eba2ad9292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:33.491328 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8eed1c-a59f-4c96-8c77-08eba2ad9292 service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:33.500424 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1423/2872] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:36.660684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc232f0-ecd7-4e72-bf3b-9f23caa7df37 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:36.723385 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc232f0-ecd7-4e72-bf3b-9f23caa7df37 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:36.727913 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:36.728784 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:36.729071 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:36.729362 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] PATH_INFO: `/v3/users/bd97191264094d37aafd84daade95a8b` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:36.730534 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd97191264094d37aafd84daade95a8b)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:36.732464 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:36.748675 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:36.897747 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd97191264094d37aafd84daade95a8b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:36.898420 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd97191264094d37aafd84daade95a8b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:36.905295 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd97191264094d37aafd84daade95a8b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:36.917263 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd97191264094d37aafd84daade95a8b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:36.964476 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd97191264094d37aafd84daade95a8b'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:36.970596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1450/2873] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:36 2022] DELETE /identity/v3/users/bd97191264094d37aafd84daade95a8b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:36.975157 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de9793-4fe5-497f-9cac-fdd690fcf0fe None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:36.977401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de9793-4fe5-497f-9cac-fdd690fcf0fe None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:36.977401 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de9793-4fe5-497f-9cac-fdd690fcf0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:36.993318 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-52de9793-4fe5-497f-9cac-fdd690fcf0fe None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:36.994553 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-52de9793-4fe5-497f-9cac-fdd690fcf0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:37.052779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1424/2874] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:37.062368 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:37.063834 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:37.120045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666eae1-9207-4476-99f8-fa42b2a4757a admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:37.123750 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2c272-acbe-47d7-ba90-4b0f2ee3288a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:37.124391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c272-acbe-47d7-ba90-4b0f2ee3288a service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:37.124679 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c272-acbe-47d7-ba90-4b0f2ee3288a service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:37.124961 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2c272-acbe-47d7-ba90-4b0f2ee3288a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:37.134236 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2c272-acbe-47d7-ba90-4b0f2ee3288a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:37.136175 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2c272-acbe-47d7-ba90-4b0f2ee3288a service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:37.145557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1451/2875] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:37.541462 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de9793-4fe5-497f-9cac-fdd690fcf0fe None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:37.542902 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-52de9793-4fe5-497f-9cac-fdd690fcf0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:37.555456 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de9793-4fe5-497f-9cac-fdd690fcf0fe None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:37.561684 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:37.567610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:37.567610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:37.567610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] PATH_INFO: `/v3/projects/ab2daf36d91549abae621d21f5f05689` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:37.567610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab2daf36d91549abae621d21f5f05689)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:37.567610 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:37.690868 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab2daf36d91549abae621d21f5f05689', 'request_id': 'req-1028d790-94b7-4388-b08f-9528c249822f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:37.696356 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab2daf36d91549abae621d21f5f05689', 'request_id': 'req-1028d790-94b7-4388-b08f-9528c249822f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:37.821466 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab2daf36d91549abae621d21f5f05689', 'request_id': 'req-1028d790-94b7-4388-b08f-9528c249822f'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:37.833473 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] The token cache is being invalidate because project ab2daf36d91549abae621d21f5f05689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:37.833764 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:37.834433 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:37.835974 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1425/2876] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:37 2022] DELETE /identity/v3/projects/ab2daf36d91549abae621d21f5f05689 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:37.840565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2c272-acbe-47d7-ba90-4b0f2ee3288a service neutron] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:37.850518 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2c272-acbe-47d7-ba90-4b0f2ee3288a service neutron] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:37.906959 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:37.908553 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:37.908835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:37.909113 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] PATH_INFO: `/v3/users/3d9d2a23df6d463fb06cbc89df586b42` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:37.910331 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9d2a23df6d463fb06cbc89df586b42)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:37.912245 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:38.109176 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:38.114903 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9d2a23df6d463fb06cbc89df586b42'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.115565 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9d2a23df6d463fb06cbc89df586b42'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.123015 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9d2a23df6d463fb06cbc89df586b42'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.135687 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9d2a23df6d463fb06cbc89df586b42'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.203893 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-9122a65e-abaf-49a3-9169-25444fd3247f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9d2a23df6d463fb06cbc89df586b42'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.210069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1452/2877] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:37 2022] DELETE /identity/v3/users/3d9d2a23df6d463fb06cbc89df586b42 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:38.640000 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:38.641507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:38.655189 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028d790-94b7-4388-b08f-9528c249822f admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:38.697256 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:38.713329 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:38.714248 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:38.714507 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:38.714779 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] PATH_INFO: `/v3/projects/b83034b270224c338c0186374d40f73e` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:38.716038 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83034b270224c338c0186374d40f73e)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:38.717977 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:38.826636 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83034b270224c338c0186374d40f73e', 'request_id': 'req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.831978 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83034b270224c338c0186374d40f73e', 'request_id': 'req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.894148 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83034b270224c338c0186374d40f73e', 'request_id': 'req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.906295 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] The token cache is being invalidate because project b83034b270224c338c0186374d40f73e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:38.908560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.908560 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-5cf750cb-48f7-4b53-88a2-447f48dd9ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:38.908789 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1426/2878] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:38 2022] DELETE /identity/v3/projects/b83034b270224c338c0186374d40f73e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:38.913728 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b2e50-beb0-424f-b375-cd2ead3de916 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:38.914226 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b2e50-beb0-424f-b375-cd2ead3de916 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:38.914336 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b2e50-beb0-424f-b375-cd2ead3de916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:38.934209 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-eb1b2e50-beb0-424f-b375-cd2ead3de916 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:38.935475 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-eb1b2e50-beb0-424f-b375-cd2ead3de916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:38.994047 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1453/2879] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:39.002061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204c4e0-1c85-4a4e-9b57-4689169a91df None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:39.002353 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204c4e0-1c85-4a4e-9b57-4689169a91df None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:39.002702 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204c4e0-1c85-4a4e-9b57-4689169a91df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:39.021061 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-7204c4e0-1c85-4a4e-9b57-4689169a91df None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:39.025407 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-7204c4e0-1c85-4a4e-9b57-4689169a91df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:39.031244 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1427/2880] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:39.035287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b2e50-beb0-424f-b375-cd2ead3de916 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:39.045551 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b2e50-beb0-424f-b375-cd2ead3de916 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:39.050190 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6960-c5a7-40af-af69-577249aac083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:39.050835 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6960-c5a7-40af-af69-577249aac083 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:39.051124 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6960-c5a7-40af-af69-577249aac083 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:39.051408 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6960-c5a7-40af-af69-577249aac083 admin admin] PATH_INFO: `/v3/domains` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:39.052160 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6960-c5a7-40af-af69-577249aac083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87026) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 10:09:39.052507 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6960-c5a7-40af-af69-577249aac083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:39.054224 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6960-c5a7-40af-af69-577249aac083 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:39.065824 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1454/2881] 10.222.0.42 () {58 vars in 1152 bytes} [Fri Jun 3 10:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 10:09:39.070150 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204c4e0-1c85-4a4e-9b57-4689169a91df None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:39.079607 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204c4e0-1c85-4a4e-9b57-4689169a91df None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:39.084411 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a14d3a-3eea-40cd-99d1-b88953d6587a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:39.085051 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a14d3a-3eea-40cd-99d1-b88953d6587a admin admin] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:39.085340 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a14d3a-3eea-40cd-99d1-b88953d6587a admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:39.085614 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a14d3a-3eea-40cd-99d1-b88953d6587a admin admin] PATH_INFO: `/v3/projects` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:39.086320 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a14d3a-3eea-40cd-99d1-b88953d6587a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:39.088784 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a14d3a-3eea-40cd-99d1-b88953d6587a admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:39.217421 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1428/2882] 10.222.0.42 () {60 vars in 1152 bytes} [Fri Jun 3 10:09:39 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:39.221536 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6960-c5a7-40af-af69-577249aac083 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:39.232024 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6960-c5a7-40af-af69-577249aac083 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:39.288010 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9a376-3429-499d-b206-464935f2df9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:39.288671 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9a376-3429-499d-b206-464935f2df9b admin admin] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:39.288960 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9a376-3429-499d-b206-464935f2df9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:39.289242 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9a376-3429-499d-b206-464935f2df9b admin admin] PATH_INFO: `/v3/users` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:39.289964 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9a376-3429-499d-b206-464935f2df9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:39.292566 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9a376-3429-499d-b206-464935f2df9b admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:39.387773 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1455/2883] 10.222.0.42 () {60 vars in 1143 bytes} [Fri Jun 3 10:09:39 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 10:09:39.392047 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a14d3a-3eea-40cd-99d1-b88953d6587a admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:39.403105 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a14d3a-3eea-40cd-99d1-b88953d6587a admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:39.407822 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fc761-acb7-442b-a8d4-328bc0d44a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:39.408481 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fc761-acb7-442b-a8d4-328bc0d44a48 admin admin] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:39.408766 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fc761-acb7-442b-a8d4-328bc0d44a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:39.409046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fc761-acb7-442b-a8d4-328bc0d44a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:39.409770 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fc761-acb7-442b-a8d4-328bc0d44a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:39.411406 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fc761-acb7-442b-a8d4-328bc0d44a48 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:39.421708 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1429/2884] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:39.425480 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9a376-3429-499d-b206-464935f2df9b admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:39.435736 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9a376-3429-499d-b206-464935f2df9b admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:39.440635 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae62a24-cb33-4a5a-a329-a64c055299a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:39.441287 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae62a24-cb33-4a5a-a329-a64c055299a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:39.441574 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae62a24-cb33-4a5a-a329-a64c055299a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:39.441856 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae62a24-cb33-4a5a-a329-a64c055299a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:39.442612 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae62a24-cb33-4a5a-a329-a64c055299a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:39.444246 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae62a24-cb33-4a5a-a329-a64c055299a8 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:39.456460 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1456/2885] 10.222.0.42 () {58 vars in 1121 bytes} [Fri Jun 3 10:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 10:09:39.460339 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fc761-acb7-442b-a8d4-328bc0d44a48 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:39.470967 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fc761-acb7-442b-a8d4-328bc0d44a48 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:39.475557 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bec8b-d8bc-4c9c-b86b-8ab3c3f59ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:39.476428 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bec8b-d8bc-4c9c-b86b-8ab3c3f59ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:39.476718 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bec8b-d8bc-4c9c-b86b-8ab3c3f59ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:39.476995 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bec8b-d8bc-4c9c-b86b-8ab3c3f59ce4 admin admin] PATH_INFO: `/v3/projects/222bec6d7d4c4aa8a2ae3c0dcf8122b1/users/c714d9949d34418595c8643ac4f43626/roles/7bcc1da1e79942c0a21920d35347bf06` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:39.491597 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bec8b-d8bc-4c9c-b86b-8ab3c3f59ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=222bec6d7d4c4aa8a2ae3c0dcf8122b1, user_id=c714d9949d34418595c8643ac4f43626, role_id=7bcc1da1e79942c0a21920d35347bf06)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:39.494336 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bec8b-d8bc-4c9c-b86b-8ab3c3f59ce4 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:39.587514 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1430/2886] 10.222.0.42 () {60 vars in 1482 bytes} [Fri Jun 3 10:09:39 2022] PUT /identity/v3/projects/222bec6d7d4c4aa8a2ae3c0dcf8122b1/users/c714d9949d34418595c8643ac4f43626/roles/7bcc1da1e79942c0a21920d35347bf06 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:39.599648 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f535-964e-42b0-b2fe-d77f2ff561f9 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:39.599940 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f535-964e-42b0-b2fe-d77f2ff561f9 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:39.600254 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f535-964e-42b0-b2fe-d77f2ff561f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:39.626476 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-6f74f535-964e-42b0-b2fe-d77f2ff561f9 None None] MFA Rules not processed for user `c714d9949d34418595c8643ac4f43626`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:39.627732 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-6f74f535-964e-42b0-b2fe-d77f2ff561f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:39.638691 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:39.693391 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1457/2887] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:39.704645 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bec8b-d8bc-4c9c-b86b-8ab3c3f59ce4 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:39.706359 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-162bec8b-d8bc-4c9c-b86b-8ab3c3f59ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:39.764290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bec8b-d8bc-4c9c-b86b-8ab3c3f59ce4 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:39.767844 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b193be8-4f82-4c93-b014-ae650b102c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aee48298b3834984a12fe2fe0a6f9a77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:39.768487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b193be8-4f82-4c93-b014-ae650b102c1e service cinder] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:39.768777 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b193be8-4f82-4c93-b014-ae650b102c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:39.769068 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b193be8-4f82-4c93-b014-ae650b102c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:39.777163 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b193be8-4f82-4c93-b014-ae650b102c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:39.778633 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b193be8-4f82-4c93-b014-ae650b102c1e service cinder] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:39.787350 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1431/2888] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:43.755594 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74f535-964e-42b0-b2fe-d77f2ff561f9 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:43.816900 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74f535-964e-42b0-b2fe-d77f2ff561f9 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:43.821371 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:43.822260 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:43.823140 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:43.823436 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] PATH_INFO: `/v3/users/c714d9949d34418595c8643ac4f43626` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:43.824616 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c714d9949d34418595c8643ac4f43626)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:43.826613 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:43.995026 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c714d9949d34418595c8643ac4f43626'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:43.995667 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c714d9949d34418595c8643ac4f43626'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:44.003040 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c714d9949d34418595c8643ac4f43626'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:44.015512 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c714d9949d34418595c8643ac4f43626'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:44.112499 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c714d9949d34418595c8643ac4f43626'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:44.118646 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1458/2889] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:09:43 2022] DELETE /identity/v3/users/c714d9949d34418595c8643ac4f43626 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:44.123441 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff3e14-3d3b-4ba9-b7b3-80d1d9d98788 None None] REQUEST_METHOD: `POST` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:44.123952 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff3e14-3d3b-4ba9-b7b3-80d1d9d98788 None None] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:44.124080 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff3e14-3d3b-4ba9-b7b3-80d1d9d98788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:44.141659 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.auth.core [None req-31ff3e14-3d3b-4ba9-b7b3-80d1d9d98788 None None] MFA Rules not processed for user `e68f65e605c34489879af9763cddb11c`. Rule list: `[]` (Enabled: `True`). {{(pid=87025) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:44.142876 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-31ff3e14-3d3b-4ba9-b7b3-80d1d9d98788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:44.147836 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:44.201963 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1432/2890] 10.222.0.42 () {58 vars in 957 bytes} [Fri Jun 3 10:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:44.211557 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:44.213127 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:44.247888 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:44.271325 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc327e93-ccc4-42f0-86f3-23733fdb220d admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:44.274917 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17634cf4-0113-4a5b-9135-ce180acf6322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '362797538cbd48dc9c73b09b0548959e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:44.275562 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17634cf4-0113-4a5b-9135-ce180acf6322 service neutron] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:44.275850 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17634cf4-0113-4a5b-9135-ce180acf6322 service neutron] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:44.276170 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17634cf4-0113-4a5b-9135-ce180acf6322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:44.285120 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17634cf4-0113-4a5b-9135-ce180acf6322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:44.286596 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17634cf4-0113-4a5b-9135-ce180acf6322 service neutron] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:44.296060 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1459/2891] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:09:44.707205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff3e14-3d3b-4ba9-b7b3-80d1d9d98788 None None] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:44.708703 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-31ff3e14-3d3b-4ba9-b7b3-80d1d9d98788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:44.722730 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff3e14-3d3b-4ba9-b7b3-80d1d9d98788 None None] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:44.727487 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:44.728391 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:44.729238 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:44.730083 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] PATH_INFO: `/v3/projects/222bec6d7d4c4aa8a2ae3c0dcf8122b1` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:44.731562 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=222bec6d7d4c4aa8a2ae3c0dcf8122b1)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:44.733678 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:44.981561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '222bec6d7d4c4aa8a2ae3c0dcf8122b1', 'request_id': 'req-4d2e466d-1dcd-4626-be31-df553135e2f1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:44.987071 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '222bec6d7d4c4aa8a2ae3c0dcf8122b1', 'request_id': 'req-4d2e466d-1dcd-4626-be31-df553135e2f1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:45.061170 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '222bec6d7d4c4aa8a2ae3c0dcf8122b1', 'request_id': 'req-4d2e466d-1dcd-4626-be31-df553135e2f1'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:45.072688 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] The token cache is being invalidate because project 222bec6d7d4c4aa8a2ae3c0dcf8122b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:09:45.072970 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:45.073621 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:09:45.075203 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1433/2892] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:09:44 2022] DELETE /identity/v3/projects/222bec6d7d4c4aa8a2ae3c0dcf8122b1 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:09:54.472661 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=12 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:55.069419 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:09:58.500095 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5db7d8-986b-4e88-b404-73e77936e408 None None] REQUEST_METHOD: `POST` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:58.500743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5db7d8-986b-4e88-b404-73e77936e408 None None] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:58.500743 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5db7d8-986b-4e88-b404-73e77936e408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:58.518899 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.auth.core [None req-aa5db7d8-986b-4e88-b404-73e77936e408 None None] MFA Rules not processed for user `fd85838dd97c4d74995302afe060388c`. Rule list: `[]` (Enabled: `True`). {{(pid=87026) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 10:09:58.520085 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aa5db7d8-986b-4e88-b404-73e77936e408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:58.574279 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1460/2893] 10.222.0.42 () {58 vars in 967 bytes} [Fri Jun 3 10:09:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 10:09:58.613325 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:09:58.614808 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:09:58.681804 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e466d-1dcd-4626-be31-df553135e2f1 admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:09:58.686930 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2829c1c-6219-43f5-a29e-3e5ec9d295e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37186dc0010247cd936f160b5eb0298b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:09:58.687574 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829c1c-6219-43f5-a29e-3e5ec9d295e4 service swift] REQUEST_METHOD: `GET` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:09:58.687865 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829c1c-6219-43f5-a29e-3e5ec9d295e4 service swift] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:09:58.688144 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829c1c-6219-43f5-a29e-3e5ec9d295e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:09:58.696027 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2829c1c-6219-43f5-a29e-3e5ec9d295e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:09:58.697627 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2829c1c-6219-43f5-a29e-3e5ec9d295e4 service swift] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:09:58.705976 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1434/2894] 10.222.0.42 () {60 vars in 1384 bytes} [Fri Jun 3 10:09:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 10:10:06.655924 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5db7d8-986b-4e88-b404-73e77936e408 None None] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:10:06.663783 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-aa5db7d8-986b-4e88-b404-73e77936e408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:10:06.723889 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5db7d8-986b-4e88-b404-73e77936e408 None None] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:10:06.728503 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:10:06.729398 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:10:06.729682 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:10:06.729991 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] PATH_INFO: `/v3/users/0b94fa871aee4c35851e6e939838d2f6` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:10:06.731738 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b94fa871aee4c35851e6e939838d2f6)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:10:06.733713 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:10:06.930048 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b94fa871aee4c35851e6e939838d2f6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:06.930637 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b94fa871aee4c35851e6e939838d2f6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:06.937669 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b94fa871aee4c35851e6e939838d2f6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:06.950171 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b94fa871aee4c35851e6e939838d2f6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.032390 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b94fa871aee4c35851e6e939838d2f6'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.038496 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1461/2895] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:10:06 2022] DELETE /identity/v3/users/0b94fa871aee4c35851e6e939838d2f6 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:10:07.325828 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2829c1c-6219-43f5-a29e-3e5ec9d295e4 service swift] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:10:07.336561 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2829c1c-6219-43f5-a29e-3e5ec9d295e4 service swift] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:10:07.392640 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:10:07.393511 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:10:07.393795 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:10:07.394099 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] PATH_INFO: `/v3/projects/bc1aca15157e45bf95284345dcabf101` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:10:07.395290 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1aca15157e45bf95284345dcabf101)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:10:07.397221 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:10:07.558791 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1aca15157e45bf95284345dcabf101', 'request_id': 'req-8913d317-3514-493c-919f-087f4ac81f0d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.565440 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1aca15157e45bf95284345dcabf101', 'request_id': 'req-8913d317-3514-493c-919f-087f4ac81f0d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.632855 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1aca15157e45bf95284345dcabf101', 'request_id': 'req-8913d317-3514-493c-919f-087f4ac81f0d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.645495 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] The token cache is being invalidate because project bc1aca15157e45bf95284345dcabf101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:10:07.645787 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.646472 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.648025 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1435/2896] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:10:07 2022] DELETE /identity/v3/projects/bc1aca15157e45bf95284345dcabf101 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:10:07.652337 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:10:07.653968 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:10:07.667891 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e647024-5ad7-48c5-a750-d020c9b8d001 admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:10:07.688684 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:10:07.726004 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:10:07.726883 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:10:07.727166 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:10:07.727439 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] PATH_INFO: `/v3/users/c151f74afd1d47838ef11241f3d93f06` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:10:07.728622 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=c151f74afd1d47838ef11241f3d93f06)` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:10:07.730573 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:10:07.914470 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c151f74afd1d47838ef11241f3d93f06'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.915045 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c151f74afd1d47838ef11241f3d93f06'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.922471 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c151f74afd1d47838ef11241f3d93f06'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:07.934473 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c151f74afd1d47838ef11241f3d93f06'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:08.014826 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.notifications [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c151f74afd1d47838ef11241f3d93f06'} {{(pid=87026) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:08.020967 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1462/2897] 10.222.0.42 () {58 vars in 1223 bytes} [Fri Jun 3 10:10:07 2022] DELETE /identity/v3/users/c151f74afd1d47838ef11241f3d93f06 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:10:08.291375 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] Authenticating user token {{(pid=87025) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:10:08.296291 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.fernet_utils [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87025) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:10:08.305806 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d317-3514-493c-919f-087f4ac81f0d admin admin] Validating token access rules against request {{(pid=87025) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:10:08.323204 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:10:08.362547 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e68f65e605c34489879af9763cddb11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53d2a61c9fa84503ab8a6c4eb2d3f27e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87025) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:10:08.363394 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:10:08.363671 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] SCRIPT_NAME: `/identity` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:10:08.363940 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] PATH_INFO: `/v3/projects/10c669a8ac59456fbf4e8bac2135a2a1` {{(pid=87025) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:10:08.365100 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] RBAC: Authorizing `identity:delete_project(project_id=10c669a8ac59456fbf4e8bac2135a2a1)` {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:10:08.367046 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] RBAC: Authorization granted {{(pid=87025) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:10:08.494669 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10c669a8ac59456fbf4e8bac2135a2a1', 'request_id': 'req-a2986d16-85d5-41ef-b010-f078744db86d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:08.500205 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10c669a8ac59456fbf4e8bac2135a2a1', 'request_id': 'req-a2986d16-85d5-41ef-b010-f078744db86d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:08.579395 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10c669a8ac59456fbf4e8bac2135a2a1', 'request_id': 'req-a2986d16-85d5-41ef-b010-f078744db86d'} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:08.590534 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] The token cache is being invalidate because project 10c669a8ac59456fbf4e8bac2135a2a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87025) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 10:10:08.590821 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:08.591450 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG keystone.notifications [None req-a2986d16-85d5-41ef-b010-f078744db86d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87025) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 10:10:08.593006 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: [pid: 87025|app: 0|req: 1436/2898] 10.222.0.42 () {58 vars in 1232 bytes} [Fri Jun 3 10:10:08 2022] DELETE /identity/v3/projects/10c669a8ac59456fbf4e8bac2135a2a1 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 10:10:18.017301 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:10:18.814363 dv-cinder-844563-1-iscsi devstack@keystone.service[87025]: DEBUG dbcounter [-] [87025] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87025) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 10:12:23.916069 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] Authenticating user token {{(pid=87026) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 10:12:23.917451 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:12:24.059587 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c92935-9375-40d5-b52c-5324ddb016ed admin admin] Validating token access rules against request {{(pid=87026) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 10:12:24.117568 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10538a-1d63-4e7d-9268-23b4e2969f2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '566c0c4bdcaf4c13b060af4bbff5bc04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c179ab83d804662a720cedf0932336f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87026) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 10:12:24.118239 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10538a-1d63-4e7d-9268-23b4e2969f2d service placement] REQUEST_METHOD: `GET` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 10:12:24.118527 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10538a-1d63-4e7d-9268-23b4e2969f2d service placement] SCRIPT_NAME: `/identity` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 10:12:24.118806 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10538a-1d63-4e7d-9268-23b4e2969f2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87026) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 10:12:24.120584 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.fernet_utils [None req-1b10538a-1d63-4e7d-9268-23b4e2969f2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87026) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 10:12:24.132237 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10538a-1d63-4e7d-9268-23b4e2969f2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 10:12:24.133802 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10538a-1d63-4e7d-9268-23b4e2969f2d service placement] RBAC: Authorization granted {{(pid=87026) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 10:12:24.144344 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: [pid: 87026|app: 0|req: 1463/2899] 10.222.0.42 () {60 vars in 1365 bytes} [Fri Jun 3 10:12:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 10:12:34.139490 dv-cinder-844563-1-iscsi devstack@keystone.service[87026]: DEBUG dbcounter [-] [87026] Writing DB stats keystone:SELECT=89 {{(pid=87026) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}